2024-09-11T07:41:29.188 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-11T07:41:29.202 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-11T07:41:29.289 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901238 branch: wip-guits-quincy-2024-09-10-2008-quincy description: orch:cephadm/smoke/{0-distro/centos_9.stream 0-nvme-loop agent/on fixed-2 mon_election/classic start} email: gabrioux@ibm.com first_in_suite: false job_id: '7901238' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-guits-quincy-2024-09-10-2008-quincy ceph: conf: global: mon election default strategy: 1 mgr: debug mgr: 20 debug ms: 1 mgr/cephadm/use_agent: true mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug ms: 1 debug osd: 20 osd shutdown pgref assert: true flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - MON_DOWN - mons down - mon down - out of quorum sha1: d0e6828a2016d48cf25ad84064e50742bb1c39b9 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: d0e6828a2016d48cf25ad84064e50742bb1c39b9 kernel: hwe: true selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: wip-guits-quincy-2024-09-10-2008-quincy sha1: d0e6828a2016d48cf25ad84064e50742bb1c39b9 owner: scheduled_gabrioux@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - ceph.rgw.foo.a - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - grafana.a - node-exporter.b - ceph.iscsi.iscsi.a seed: 2282 sha1: d0e6828a2016d48cf25ad84064e50742bb1c39b9 sleep_before_teardown: 0 subset: 1/8 suite: orch:cephadm suite_branch: wip-guits-quincy-2024-09-10-2008-quincy suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_d0e6828a2016d48cf25ad84064e50742bb1c39b9/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: d0e6828a2016d48cf25ad84064e50742bb1c39b9 targets: smithi110.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDNE3OG/V66gzzkdddYvp3sJthMFGgKGNmQdxc9X88YGUH6aw5q2cbfT+Dqgsykz/Gt7OHxghEu1H5CbDSmZDT0= smithi139.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE8ncFlUL29MraV9BR+O864L+rsPenBjJ35VvSCHCKJf5cNiEwK+CtPze9HTBRygk+zAm03uMHMLPPvb2N59EMI= tasks: - pexec: all: - sudo dnf remove nvme-cli -y - sudo dnf install nvmetcli nvme-cli -y - nvme_loop: null - cephadm: conf: mgr: debug mgr: 20 debug ms: 1 - cephadm.shell: mon.a: - stat -c '%u %g' /var/log/ceph | grep '167 167' - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - ceph orch ls --format yaml - ceph orch ls | grep '^osd ' teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-11_06:53:43 tube: smithi user: gabrioux verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-11T07:41:29.289 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_d0e6828a2016d48cf25ad84064e50742bb1c39b9/qa; will attempt to use it 2024-09-11T07:41:29.290 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_d0e6828a2016d48cf25ad84064e50742bb1c39b9/qa/tasks 2024-09-11T07:41:29.290 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-11T07:41:29.291 INFO:teuthology.task.internal:Checking packages... 2024-09-11T07:41:29.312 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd0e6828a2016d48cf25ad84064e50742bb1c39b9' 2024-09-11T07:41:29.312 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-11T07:41:29.312 INFO:teuthology.packaging:ref: None 2024-09-11T07:41:29.312 INFO:teuthology.packaging:tag: None 2024-09-11T07:41:29.312 INFO:teuthology.packaging:branch: wip-guits-quincy-2024-09-10-2008-quincy 2024-09-11T07:41:29.312 INFO:teuthology.packaging:sha1: d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T07:41:29.312 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-guits-quincy-2024-09-10-2008-quincy 2024-09-11T07:41:29.453 INFO:teuthology.task.internal:Found packages for ceph version 17.2.7-1623.gd0e6828a 2024-09-11T07:41:29.455 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-11T07:41:29.470 INFO:teuthology.task.internal:no buildpackages task found 2024-09-11T07:41:29.470 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-11T07:41:29.534 INFO:teuthology.task.internal:Saving configuration 2024-09-11T07:41:29.545 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-11T07:41:29.552 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-11T07:41:29.578 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi110.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901238', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-11 07:36:37.770633', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDNE3OG/V66gzzkdddYvp3sJthMFGgKGNmQdxc9X88YGUH6aw5q2cbfT+Dqgsykz/Gt7OHxghEu1H5CbDSmZDT0='} 2024-09-11T07:41:29.603 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi139.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901238', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-11 07:36:37.769139', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE8ncFlUL29MraV9BR+O864L+rsPenBjJ35VvSCHCKJf5cNiEwK+CtPze9HTBRygk+zAm03uMHMLPPvb2N59EMI='} 2024-09-11T07:41:29.604 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-11T07:41:29.610 INFO:teuthology.task.internal:roles: ubuntu@smithi110.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2024-09-11T07:41:29.610 INFO:teuthology.task.internal:roles: ubuntu@smithi139.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2024-09-11T07:41:29.610 INFO:teuthology.run_tasks:Running task console_log... 2024-09-11T07:41:29.667 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7efe2f183ac0>, signals=[15]) 2024-09-11T07:41:29.667 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-11T07:41:29.673 INFO:teuthology.task.internal:Opening connections... 2024-09-11T07:41:29.674 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi110.front.sepia.ceph.com 2024-09-11T07:41:29.675 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T07:41:29.756 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi139.front.sepia.ceph.com 2024-09-11T07:41:29.757 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T07:41:29.872 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-11T07:41:29.879 DEBUG:teuthology.orchestra.run.smithi110:> uname -m 2024-09-11T07:41:29.897 INFO:teuthology.orchestra.run.smithi110.stdout:x86_64 2024-09-11T07:41:29.897 DEBUG:teuthology.orchestra.run.smithi110:> cat /etc/os-release 2024-09-11T07:41:29.956 INFO:teuthology.orchestra.run.smithi110.stdout:NAME="CentOS Stream" 2024-09-11T07:41:29.956 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION="9" 2024-09-11T07:41:29.956 INFO:teuthology.orchestra.run.smithi110.stdout:ID="centos" 2024-09-11T07:41:29.956 INFO:teuthology.orchestra.run.smithi110.stdout:ID_LIKE="rhel fedora" 2024-09-11T07:41:29.956 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION_ID="9" 2024-09-11T07:41:29.956 INFO:teuthology.orchestra.run.smithi110.stdout:PLATFORM_ID="platform:el9" 2024-09-11T07:41:29.956 INFO:teuthology.orchestra.run.smithi110.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-11T07:41:29.956 INFO:teuthology.orchestra.run.smithi110.stdout:ANSI_COLOR="0;31" 2024-09-11T07:41:29.957 INFO:teuthology.orchestra.run.smithi110.stdout:LOGO="fedora-logo-icon" 2024-09-11T07:41:29.957 INFO:teuthology.orchestra.run.smithi110.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-11T07:41:29.957 INFO:teuthology.orchestra.run.smithi110.stdout:HOME_URL="https://centos.org/" 2024-09-11T07:41:29.957 INFO:teuthology.orchestra.run.smithi110.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-11T07:41:29.957 INFO:teuthology.orchestra.run.smithi110.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-11T07:41:29.957 INFO:teuthology.orchestra.run.smithi110.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-11T07:41:29.958 INFO:teuthology.lock.ops:Updating smithi110.front.sepia.ceph.com on lock server 2024-09-11T07:41:29.975 DEBUG:teuthology.orchestra.run.smithi139:> uname -m 2024-09-11T07:41:29.993 INFO:teuthology.orchestra.run.smithi139.stdout:x86_64 2024-09-11T07:41:29.993 DEBUG:teuthology.orchestra.run.smithi139:> cat /etc/os-release 2024-09-11T07:41:30.051 INFO:teuthology.orchestra.run.smithi139.stdout:NAME="CentOS Stream" 2024-09-11T07:41:30.051 INFO:teuthology.orchestra.run.smithi139.stdout:VERSION="9" 2024-09-11T07:41:30.051 INFO:teuthology.orchestra.run.smithi139.stdout:ID="centos" 2024-09-11T07:41:30.052 INFO:teuthology.orchestra.run.smithi139.stdout:ID_LIKE="rhel fedora" 2024-09-11T07:41:30.052 INFO:teuthology.orchestra.run.smithi139.stdout:VERSION_ID="9" 2024-09-11T07:41:30.052 INFO:teuthology.orchestra.run.smithi139.stdout:PLATFORM_ID="platform:el9" 2024-09-11T07:41:30.052 INFO:teuthology.orchestra.run.smithi139.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-11T07:41:30.052 INFO:teuthology.orchestra.run.smithi139.stdout:ANSI_COLOR="0;31" 2024-09-11T07:41:30.052 INFO:teuthology.orchestra.run.smithi139.stdout:LOGO="fedora-logo-icon" 2024-09-11T07:41:30.052 INFO:teuthology.orchestra.run.smithi139.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-11T07:41:30.052 INFO:teuthology.orchestra.run.smithi139.stdout:HOME_URL="https://centos.org/" 2024-09-11T07:41:30.052 INFO:teuthology.orchestra.run.smithi139.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-11T07:41:30.052 INFO:teuthology.orchestra.run.smithi139.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-11T07:41:30.052 INFO:teuthology.orchestra.run.smithi139.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-11T07:41:30.053 INFO:teuthology.lock.ops:Updating smithi139.front.sepia.ceph.com on lock server 2024-09-11T07:41:30.075 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-11T07:41:30.086 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-11T07:41:30.097 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-11T07:41:30.097 DEBUG:teuthology.orchestra.run.smithi110:> test '!' -e /home/ubuntu/cephtest 2024-09-11T07:41:30.102 DEBUG:teuthology.orchestra.run.smithi139:> test '!' -e /home/ubuntu/cephtest 2024-09-11T07:41:30.128 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-11T07:41:30.154 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-11T07:41:30.155 DEBUG:teuthology.orchestra.run.smithi110:> test -z $(ls -A /var/lib/ceph) 2024-09-11T07:41:30.162 DEBUG:teuthology.orchestra.run.smithi139:> test -z $(ls -A /var/lib/ceph) 2024-09-11T07:41:30.210 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-11T07:41:30.316 INFO:teuthology.run_tasks:Running task kernel... 2024-09-11T07:41:30.331 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-11T07:41:30.332 INFO:teuthology.task.kernel:normalize config orig: {'hwe': True} 2024-09-11T07:41:30.332 DEBUG:teuthology.task.kernel:normalized overrides {'mon.a': {'hwe': True}, 'mon.c': {'hwe': True}, 'mgr.y': {'hwe': True}, 'osd.0': {'hwe': True}, 'osd.1': {'hwe': True}, 'osd.2': {'hwe': True}, 'osd.3': {'hwe': True}, 'client.0': {'hwe': True}, 'ceph.rgw.foo.a': {'hwe': True}, 'node-exporter.a': {'hwe': True}, 'alertmanager.a': {'hwe': True}, 'mon.b': {'hwe': True}, 'mgr.x': {'hwe': True}, 'osd.4': {'hwe': True}, 'osd.5': {'hwe': True}, 'osd.6': {'hwe': True}, 'osd.7': {'hwe': True}, 'client.1': {'hwe': True}, 'prometheus.a': {'hwe': True}, 'grafana.a': {'hwe': True}, 'node-exporter.b': {'hwe': True}, 'ceph.iscsi.iscsi.a': {'hwe': True}} 2024-09-11T07:41:30.332 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro', 'hwe': True}, 'mon.b': {'kdb': 1, 'sha1': 'distro', 'hwe': True}}, timeout 300 2024-09-11T07:41:30.332 DEBUG:teuthology.orchestra.run.smithi110:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-11T07:41:30.333 DEBUG:teuthology.orchestra.run.smithi139:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-11T07:41:30.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:30.350 DEBUG:teuthology.orchestra.run.smithi139:> uname -r 2024-09-11T07:41:30.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:30.351 DEBUG:teuthology.orchestra.run.smithi110:> uname -r 2024-09-11T07:41:30.408 INFO:teuthology.orchestra.run.smithi139.stdout:5.14.0-503.el9.x86_64 2024-09-11T07:41:30.408 INFO:teuthology.task.kernel:Running kernel on smithi139: 5.14.0-503.el9.x86_64 2024-09-11T07:41:30.408 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum install -y kernel 2024-09-11T07:41:30.409 INFO:teuthology.orchestra.run.smithi110.stdout:5.14.0-503.el9.x86_64 2024-09-11T07:41:30.409 INFO:teuthology.task.kernel:Running kernel on smithi110: 5.14.0-503.el9.x86_64 2024-09-11T07:41:30.410 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum install -y kernel 2024-09-11T07:41:34.040 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - BaseOS 4.5 MB/s | 8.2 MB 00:01 2024-09-11T07:41:34.105 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - BaseOS 4.4 MB/s | 8.2 MB 00:01 2024-09-11T07:41:38.766 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - AppStream 5.6 MB/s | 20 MB 00:03 2024-09-11T07:41:38.822 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - AppStream 5.7 MB/s | 20 MB 00:03 2024-09-11T07:41:43.081 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.5 MB 00:00 2024-09-11T07:41:43.200 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.5 MB 00:00 2024-09-11T07:41:44.634 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - Extras packages 86 kB/s | 19 kB 00:00 2024-09-11T07:41:45.331 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - Extras packages 18 kB/s | 19 kB 00:01 2024-09-11T07:41:45.389 INFO:teuthology.orchestra.run.smithi110.stdout:Extra Packages for Enterprise Linux 36 MB/s | 23 MB 00:00 2024-09-11T07:41:46.260 INFO:teuthology.orchestra.run.smithi139.stdout:Extra Packages for Enterprise Linux 28 MB/s | 23 MB 00:00 2024-09-11T07:41:50.778 INFO:teuthology.orchestra.run.smithi110.stdout:lab-extras 35 kB/s | 1.7 kB 00:00 2024-09-11T07:41:51.674 INFO:teuthology.orchestra.run.smithi139.stdout:lab-extras 43 kB/s | 1.7 kB 00:00 2024-09-11T07:41:52.742 INFO:teuthology.orchestra.run.smithi110.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-11T07:41:52.743 INFO:teuthology.orchestra.run.smithi110.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-11T07:41:52.829 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-09-11T07:41:52.836 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-11T07:41:52.836 INFO:teuthology.orchestra.run.smithi110.stdout: Package Architecture Version Repository Size 2024-09-11T07:41:52.836 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-11T07:41:52.836 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2024-09-11T07:41:52.836 INFO:teuthology.orchestra.run.smithi110.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-11T07:41:52.836 INFO:teuthology.orchestra.run.smithi110.stdout:Installing dependencies: 2024-09-11T07:41:52.836 INFO:teuthology.orchestra.run.smithi110.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-11T07:41:52.836 INFO:teuthology.orchestra.run.smithi110.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-11T07:41:52.836 INFO:teuthology.orchestra.run.smithi110.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-11T07:41:52.836 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T07:41:52.836 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2024-09-11T07:41:52.836 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-11T07:41:52.837 INFO:teuthology.orchestra.run.smithi110.stdout:Install 4 Packages 2024-09-11T07:41:52.837 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T07:41:52.837 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 79 M 2024-09-11T07:41:52.837 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 126 M 2024-09-11T07:41:52.837 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2024-09-11T07:41:53.237 INFO:teuthology.orchestra.run.smithi110.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 188 kB/s | 23 kB 00:00 2024-09-11T07:41:53.630 INFO:teuthology.orchestra.run.smithi139.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-11T07:41:53.631 INFO:teuthology.orchestra.run.smithi139.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-11T07:41:53.716 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-09-11T07:41:53.724 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-11T07:41:53.725 INFO:teuthology.orchestra.run.smithi139.stdout: Package Architecture Version Repository Size 2024-09-11T07:41:53.725 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-11T07:41:53.725 INFO:teuthology.orchestra.run.smithi139.stdout:Installing: 2024-09-11T07:41:53.725 INFO:teuthology.orchestra.run.smithi139.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-11T07:41:53.725 INFO:teuthology.orchestra.run.smithi139.stdout:Installing dependencies: 2024-09-11T07:41:53.725 INFO:teuthology.orchestra.run.smithi139.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-11T07:41:53.725 INFO:teuthology.orchestra.run.smithi139.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-11T07:41:53.725 INFO:teuthology.orchestra.run.smithi139.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-11T07:41:53.725 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-11T07:41:53.725 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction Summary 2024-09-11T07:41:53.725 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-11T07:41:53.725 INFO:teuthology.orchestra.run.smithi139.stdout:Install 4 Packages 2024-09-11T07:41:53.725 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-11T07:41:53.726 INFO:teuthology.orchestra.run.smithi139.stdout:Total download size: 79 M 2024-09-11T07:41:53.726 INFO:teuthology.orchestra.run.smithi139.stdout:Installed size: 126 M 2024-09-11T07:41:53.726 INFO:teuthology.orchestra.run.smithi139.stdout:Downloading Packages: 2024-09-11T07:41:53.813 INFO:teuthology.orchestra.run.smithi110.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 22 MB/s | 16 MB 00:00 2024-09-11T07:41:53.934 INFO:teuthology.orchestra.run.smithi139.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 286 kB/s | 23 kB 00:00 2024-09-11T07:41:54.309 INFO:teuthology.orchestra.run.smithi139.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 34 MB/s | 16 MB 00:00 2024-09-11T07:41:54.421 INFO:teuthology.orchestra.run.smithi110.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 26 MB/s | 35 MB 00:01 2024-09-11T07:41:54.701 INFO:teuthology.orchestra.run.smithi139.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 41 MB/s | 35 MB 00:00 2024-09-11T07:41:55.005 INFO:teuthology.orchestra.run.smithi110.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 16 MB/s | 29 MB 00:01 2024-09-11T07:41:55.005 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:41:55.005 INFO:teuthology.orchestra.run.smithi110.stdout:Total 36 MB/s | 79 MB 00:02 2024-09-11T07:41:55.059 INFO:teuthology.orchestra.run.smithi139.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 25 MB/s | 29 MB 00:01 2024-09-11T07:41:55.060 INFO:teuthology.orchestra.run.smithi139.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:41:55.060 INFO:teuthology.orchestra.run.smithi139.stdout:Total 59 MB/s | 79 MB 00:01 2024-09-11T07:41:55.686 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2024-09-11T07:41:55.742 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction check 2024-09-11T07:41:56.035 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2024-09-11T07:41:56.035 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2024-09-11T07:41:56.108 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction check succeeded. 2024-09-11T07:41:56.108 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction test 2024-09-11T07:41:56.969 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2024-09-11T07:41:56.969 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2024-09-11T07:41:57.064 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction test succeeded. 2024-09-11T07:41:57.065 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction 2024-09-11T07:41:59.396 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2024-09-11T07:41:59.525 INFO:teuthology.orchestra.run.smithi139.stdout: Preparing : 1/1 2024-09-11T07:42:00.138 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-11T07:42:00.215 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T07:42:00.250 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-11T07:42:00.342 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T07:42:03.004 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T07:42:03.142 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T07:42:03.158 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T07:42:03.287 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T07:42:07.941 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T07:42:08.059 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:42:08.108 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T07:42:08.229 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:42:12.880 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:42:13.060 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:42:51.145 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:42:51.183 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:42:51.453 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:42:51.493 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:42:51.524 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:42:51.524 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-11T07:42:51.524 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T07:42:51.524 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T07:42:51.870 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:42:51.870 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-11T07:42:51.870 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T07:42:51.870 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T07:42:52.433 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:42:52.433 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T07:42:52.433 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2024-09-11T07:42:52.433 INFO:teuthology.orchestra.run.smithi110.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-11T07:42:52.433 INFO:teuthology.orchestra.run.smithi110.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-11T07:42:52.433 INFO:teuthology.orchestra.run.smithi110.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-11T07:42:52.433 INFO:teuthology.orchestra.run.smithi110.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-11T07:42:52.433 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T07:42:52.433 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-09-11T07:42:52.691 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:42:52.691 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-11T07:42:52.692 INFO:teuthology.orchestra.run.smithi139.stdout:Installed: 2024-09-11T07:42:52.692 INFO:teuthology.orchestra.run.smithi139.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-11T07:42:52.692 INFO:teuthology.orchestra.run.smithi139.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-11T07:42:52.692 INFO:teuthology.orchestra.run.smithi139.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-11T07:42:52.692 INFO:teuthology.orchestra.run.smithi139.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-11T07:42:52.692 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-11T07:42:52.692 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-09-11T07:42:52.796 DEBUG:teuthology.orchestra.run.smithi110:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T07:42:52.873 INFO:teuthology.orchestra.run.smithi110.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T07:42:52.874 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T07:42:52.874 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-11T07:42:52.874 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-11T07:42:52.875 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-11T07:42:52.875 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-11T07:42:52.875 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi110.front.sepia.ceph.com, path=None, version=distro) 2024-09-11T07:42:52.876 DEBUG:teuthology.orchestra.run.smithi110:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T07:42:52.933 INFO:teuthology.orchestra.run.smithi110.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T07:42:52.934 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T07:42:52.934 DEBUG:teuthology.orchestra.run.smithi110:> sudo rpm -qi grub2-tools 2024-09-11T07:42:52.965 INFO:teuthology.orchestra.run.smithi110.stdout:Name : grub2-tools 2024-09-11T07:42:52.966 INFO:teuthology.orchestra.run.smithi110.stdout:Epoch : 1 2024-09-11T07:42:52.966 INFO:teuthology.orchestra.run.smithi110.stdout:Version : 2.06 2024-09-11T07:42:52.966 INFO:teuthology.orchestra.run.smithi110.stdout:Release : 82.el9 2024-09-11T07:42:52.966 INFO:teuthology.orchestra.run.smithi110.stdout:Architecture: x86_64 2024-09-11T07:42:52.966 INFO:teuthology.orchestra.run.smithi110.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-11T07:42:52.966 INFO:teuthology.orchestra.run.smithi110.stdout:Group : Unspecified 2024-09-11T07:42:52.966 INFO:teuthology.orchestra.run.smithi110.stdout:Size : 8274670 2024-09-11T07:42:52.966 INFO:teuthology.orchestra.run.smithi110.stdout:License : GPLv3+ 2024-09-11T07:42:52.966 INFO:teuthology.orchestra.run.smithi110.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-11T07:42:52.966 INFO:teuthology.orchestra.run.smithi110.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-11T07:42:52.966 INFO:teuthology.orchestra.run.smithi110.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-11T07:42:52.966 INFO:teuthology.orchestra.run.smithi110.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-11T07:42:52.967 INFO:teuthology.orchestra.run.smithi110.stdout:Packager : builder@centos.org 2024-09-11T07:42:52.967 INFO:teuthology.orchestra.run.smithi110.stdout:Vendor : CentOS 2024-09-11T07:42:52.967 INFO:teuthology.orchestra.run.smithi110.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-11T07:42:52.967 INFO:teuthology.orchestra.run.smithi110.stdout:Summary : Support tools for GRUB. 2024-09-11T07:42:52.967 INFO:teuthology.orchestra.run.smithi110.stdout:Description : 2024-09-11T07:42:52.967 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T07:42:52.967 INFO:teuthology.orchestra.run.smithi110.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-11T07:42:52.967 INFO:teuthology.orchestra.run.smithi110.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-11T07:42:52.967 INFO:teuthology.orchestra.run.smithi110.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-11T07:42:52.967 INFO:teuthology.orchestra.run.smithi110.stdout:hardware devices. 2024-09-11T07:42:52.967 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T07:42:52.967 INFO:teuthology.orchestra.run.smithi110.stdout:This subpackage provides tools for support of all platforms. 2024-09-11T07:42:52.968 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-11T07:42:52.968 INFO:teuthology.task.kernel:Updating grub on smithi110 to boot 5.14.0-505.el9.x86_64 2024-09-11T07:42:52.968 DEBUG:teuthology.orchestra.run.smithi110:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-11T07:42:53.057 DEBUG:teuthology.orchestra.run.smithi139:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T07:42:53.146 INFO:teuthology.orchestra.run.smithi139.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T07:42:53.147 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T07:42:53.147 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-11T07:42:53.147 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-11T07:42:53.147 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-11T07:42:53.147 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-11T07:42:53.148 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi139.front.sepia.ceph.com, path=None, version=distro) 2024-09-11T07:42:53.148 DEBUG:teuthology.orchestra.run.smithi139:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T07:42:53.207 INFO:teuthology.orchestra.run.smithi139.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T07:42:53.207 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T07:42:53.207 DEBUG:teuthology.orchestra.run.smithi139:> sudo rpm -qi grub2-tools 2024-09-11T07:42:53.248 INFO:teuthology.orchestra.run.smithi139.stdout:Name : grub2-tools 2024-09-11T07:42:53.248 INFO:teuthology.orchestra.run.smithi139.stdout:Epoch : 1 2024-09-11T07:42:53.248 INFO:teuthology.orchestra.run.smithi139.stdout:Version : 2.06 2024-09-11T07:42:53.249 INFO:teuthology.orchestra.run.smithi139.stdout:Release : 82.el9 2024-09-11T07:42:53.249 INFO:teuthology.orchestra.run.smithi139.stdout:Architecture: x86_64 2024-09-11T07:42:53.249 INFO:teuthology.orchestra.run.smithi139.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-11T07:42:53.249 INFO:teuthology.orchestra.run.smithi139.stdout:Group : Unspecified 2024-09-11T07:42:53.249 INFO:teuthology.orchestra.run.smithi139.stdout:Size : 8274670 2024-09-11T07:42:53.249 INFO:teuthology.orchestra.run.smithi139.stdout:License : GPLv3+ 2024-09-11T07:42:53.249 INFO:teuthology.orchestra.run.smithi139.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-11T07:42:53.249 INFO:teuthology.orchestra.run.smithi139.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-11T07:42:53.249 INFO:teuthology.orchestra.run.smithi139.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-11T07:42:53.249 INFO:teuthology.orchestra.run.smithi139.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-11T07:42:53.249 INFO:teuthology.orchestra.run.smithi139.stdout:Packager : builder@centos.org 2024-09-11T07:42:53.249 INFO:teuthology.orchestra.run.smithi139.stdout:Vendor : CentOS 2024-09-11T07:42:53.249 INFO:teuthology.orchestra.run.smithi139.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-11T07:42:53.250 INFO:teuthology.orchestra.run.smithi139.stdout:Summary : Support tools for GRUB. 2024-09-11T07:42:53.250 INFO:teuthology.orchestra.run.smithi139.stdout:Description : 2024-09-11T07:42:53.250 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-11T07:42:53.251 INFO:teuthology.orchestra.run.smithi139.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-11T07:42:53.251 INFO:teuthology.orchestra.run.smithi139.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-11T07:42:53.251 INFO:teuthology.orchestra.run.smithi139.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-11T07:42:53.251 INFO:teuthology.orchestra.run.smithi139.stdout:hardware devices. 2024-09-11T07:42:53.251 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-11T07:42:53.251 INFO:teuthology.orchestra.run.smithi139.stdout:This subpackage provides tools for support of all platforms. 2024-09-11T07:42:53.252 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-11T07:42:53.252 INFO:teuthology.task.kernel:Updating grub on smithi139 to boot 5.14.0-505.el9.x86_64 2024-09-11T07:42:53.253 DEBUG:teuthology.orchestra.run.smithi139:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-11T07:42:53.610 INFO:teuthology.orchestra.run.smithi110.stderr:Generating grub configuration file ... 2024-09-11T07:42:53.917 INFO:teuthology.orchestra.run.smithi139.stderr:Generating grub configuration file ... 2024-09-11T07:42:55.021 INFO:teuthology.orchestra.run.smithi110.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-11T07:42:55.059 INFO:teuthology.orchestra.run.smithi110.stderr:done 2024-09-11T07:42:55.062 DEBUG:teuthology.orchestra.run.smithi110:> mktemp 2024-09-11T07:42:55.118 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/tmp.vq4Cq8mgyq 2024-09-11T07:42:55.119 DEBUG:teuthology.orchestra.run.smithi110:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.vq4Cq8mgyq 2024-09-11T07:42:55.199 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod 0666 /tmp/tmp.vq4Cq8mgyq 2024-09-11T07:42:55.239 INFO:teuthology.orchestra.run.smithi139.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-11T07:42:55.279 INFO:teuthology.orchestra.run.smithi139.stderr:done 2024-09-11T07:42:55.286 DEBUG:teuthology.orchestra.run.smithi139:> mktemp 2024-09-11T07:42:55.302 INFO:teuthology.orchestra.run.smithi139.stdout:/tmp/tmp.0mOMycMrRC 2024-09-11T07:42:55.303 DEBUG:teuthology.orchestra.run.smithi139:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.0mOMycMrRC 2024-09-11T07:42:55.337 DEBUG:teuthology.orchestra.remote:smithi110:/tmp/tmp.vq4Cq8mgyq is 6KB 2024-09-11T07:42:55.369 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 0666 /tmp/tmp.0mOMycMrRC 2024-09-11T07:42:55.385 DEBUG:teuthology.orchestra.run.smithi110:> rm -fr /tmp/tmp.vq4Cq8mgyq 2024-09-11T07:42:55.401 DEBUG:teuthology.orchestra.run.smithi110:> sudo /bin/ls /boot/loader/entries || true 2024-09-11T07:42:55.463 INFO:teuthology.orchestra.run.smithi110.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-11T07:42:55.463 INFO:teuthology.orchestra.run.smithi110.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-11T07:42:55.463 INFO:teuthology.orchestra.run.smithi110.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-11T07:42:55.463 INFO:teuthology.orchestra.run.smithi110.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-11T07:42:55.465 DEBUG:teuthology.orchestra.run.smithi110:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-11T07:42:55.505 DEBUG:teuthology.orchestra.remote:smithi139:/tmp/tmp.0mOMycMrRC is 6KB 2024-09-11T07:42:55.527 DEBUG:teuthology.orchestra.run.smithi139:> rm -fr /tmp/tmp.0mOMycMrRC 2024-09-11T07:42:55.542 DEBUG:teuthology.orchestra.run.smithi139:> sudo /bin/ls /boot/loader/entries || true 2024-09-11T07:42:55.614 INFO:teuthology.orchestra.run.smithi139.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-11T07:42:55.614 INFO:teuthology.orchestra.run.smithi139.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-11T07:42:55.614 INFO:teuthology.orchestra.run.smithi139.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-11T07:42:55.614 INFO:teuthology.orchestra.run.smithi139.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-11T07:42:55.615 DEBUG:teuthology.orchestra.run.smithi139:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-11T07:42:55.617 DEBUG:teuthology.orchestra.run.smithi110:> sudo shutdown -r now 2024-09-11T07:42:55.769 DEBUG:teuthology.orchestra.run.smithi139:> sudo shutdown -r now 2024-09-11T07:43:25.649 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-11T07:43:25.649 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi110.front.sepia.ceph.com' 2024-09-11T07:43:25.650 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T07:43:25.772 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-11T07:43:25.772 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi139.front.sepia.ceph.com' 2024-09-11T07:43:25.773 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T07:44:25.710 DEBUG:teuthology.orchestra.remote:timed out 2024-09-11T07:44:25.776 DEBUG:teuthology.orchestra.remote:timed out 2024-09-11T07:44:34.718 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi110.front.sepia.ceph.com' 2024-09-11T07:44:34.720 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T07:44:34.776 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi139.front.sepia.ceph.com' 2024-09-11T07:44:34.777 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T07:44:35.194 DEBUG:teuthology.orchestra.run.smithi139:> true 2024-09-11T07:44:35.576 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi139.front.sepia.ceph.com' 2024-09-11T07:44:35.576 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-505.el9.x86_64"... 2024-09-11T07:44:35.576 DEBUG:teuthology.orchestra.run.smithi139:> uname -r 2024-09-11T07:44:35.591 INFO:teuthology.orchestra.run.smithi139.stdout:5.14.0-505.el9.x86_64 2024-09-11T07:44:35.591 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-11T07:44:35.591 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-11T07:44:35.591 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-11T07:44:36.159 DEBUG:teuthology.orchestra.run.smithi110:> true 2024-09-11T07:44:36.592 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-11T07:44:36.593 DEBUG:teuthology.orchestra.run.smithi139:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-11T07:44:36.641 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi110.front.sepia.ceph.com' 2024-09-11T07:44:36.641 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-11T07:44:36.642 DEBUG:teuthology.orchestra.run.smithi110:> uname -r 2024-09-11T07:44:36.657 INFO:teuthology.orchestra.run.smithi110.stdout:5.14.0-505.el9.x86_64 2024-09-11T07:44:36.658 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-11T07:44:36.658 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-11T07:44:36.658 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-11T07:44:36.673 INFO:teuthology.orchestra.run.smithi139.stdout:ttyS1 2024-09-11T07:44:36.710 DEBUG:teuthology.parallel:result is None 2024-09-11T07:44:37.659 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-11T07:44:37.659 DEBUG:teuthology.orchestra.run.smithi110:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-11T07:44:37.732 INFO:teuthology.orchestra.run.smithi110.stdout:ttyS1 2024-09-11T07:44:37.755 DEBUG:teuthology.parallel:result is None 2024-09-11T07:44:37.755 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-11T07:44:37.763 INFO:teuthology.task.internal:Creating test directory... 2024-09-11T07:44:37.764 DEBUG:teuthology.orchestra.run.smithi110:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-11T07:44:37.766 DEBUG:teuthology.orchestra.run.smithi139:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-11T07:44:37.781 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-11T07:44:37.788 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-11T07:44:37.795 INFO:teuthology.task.internal:Creating archive directory... 2024-09-11T07:44:37.795 DEBUG:teuthology.orchestra.run.smithi110:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-11T07:44:37.822 DEBUG:teuthology.orchestra.run.smithi139:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-11T07:44:37.850 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-11T07:44:37.856 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-11T07:44:37.857 DEBUG:teuthology.orchestra.run.smithi110:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-11T07:44:37.885 DEBUG:teuthology.orchestra.run.smithi139:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-11T07:44:37.911 INFO:teuthology.orchestra.run.smithi110.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T07:44:37.921 INFO:teuthology.orchestra.run.smithi110.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T07:44:37.929 INFO:teuthology.orchestra.run.smithi139.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T07:44:37.940 INFO:teuthology.orchestra.run.smithi139.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T07:44:37.941 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-11T07:44:37.949 INFO:teuthology.task.internal:Configuring sudo... 2024-09-11T07:44:37.950 DEBUG:teuthology.orchestra.run.smithi110:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-11T07:44:37.966 DEBUG:teuthology.orchestra.run.smithi139:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-11T07:44:38.010 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-11T07:44:38.019 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-11T07:44:38.019 DEBUG:teuthology.orchestra.run.smithi110:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-11T07:44:38.033 DEBUG:teuthology.orchestra.run.smithi139:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-11T07:44:38.069 DEBUG:teuthology.orchestra.run.smithi110:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T07:44:38.128 DEBUG:teuthology.orchestra.run.smithi110:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T07:44:38.173 DEBUG:teuthology.orchestra.run.smithi110:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T07:44:38.253 DEBUG:teuthology.orchestra.run.smithi110:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T07:44:38.295 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-11T07:44:38.296 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-11T07:44:38.363 DEBUG:teuthology.orchestra.run.smithi139:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T07:44:38.405 DEBUG:teuthology.orchestra.run.smithi139:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T07:44:38.454 DEBUG:teuthology.orchestra.run.smithi139:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T07:44:38.542 DEBUG:teuthology.orchestra.run.smithi139:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T07:44:38.586 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-11T07:44:38.586 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-11T07:44:38.654 DEBUG:teuthology.orchestra.run.smithi110:> sudo service rsyslog restart 2024-09-11T07:44:38.656 DEBUG:teuthology.orchestra.run.smithi139:> sudo service rsyslog restart 2024-09-11T07:44:38.714 INFO:teuthology.orchestra.run.smithi110.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T07:44:38.744 INFO:teuthology.orchestra.run.smithi139.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T07:44:39.037 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-11T07:44:39.045 INFO:teuthology.task.internal:Starting timer... 2024-09-11T07:44:39.045 INFO:teuthology.run_tasks:Running task pcp... 2024-09-11T07:44:39.055 INFO:teuthology.run_tasks:Running task selinux... 2024-09-11T07:44:39.063 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-09-11T07:44:39.064 DEBUG:teuthology.orchestra.run.smithi110:> sudo service auditd rotate 2024-09-11T07:44:39.107 INFO:teuthology.orchestra.run.smithi110.stdout:Rotating logs: 2024-09-11T07:44:39.109 DEBUG:teuthology.orchestra.run.smithi139:> sudo service auditd rotate 2024-09-11T07:44:39.172 INFO:teuthology.orchestra.run.smithi139.stdout:Rotating logs: 2024-09-11T07:44:39.174 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-11T07:44:39.174 DEBUG:teuthology.orchestra.run.smithi110:> /usr/sbin/getenforce 2024-09-11T07:44:39.191 INFO:teuthology.orchestra.run.smithi110.stdout:Permissive 2024-09-11T07:44:39.191 DEBUG:teuthology.orchestra.run.smithi139:> /usr/sbin/getenforce 2024-09-11T07:44:39.211 INFO:teuthology.orchestra.run.smithi139.stdout:Permissive 2024-09-11T07:44:39.212 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi110.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi139.front.sepia.ceph.com': 'permissive'} 2024-09-11T07:44:39.212 DEBUG:teuthology.orchestra.run.smithi110:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-11T07:44:39.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:39.256 DEBUG:teuthology.orchestra.run.smithi139:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-11T07:44:39.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:39.280 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-11T07:44:39.280 DEBUG:teuthology.orchestra.run.smithi110:> sudo /usr/sbin/setenforce permissive 2024-09-11T07:44:39.328 DEBUG:teuthology.orchestra.run.smithi139:> sudo /usr/sbin/setenforce permissive 2024-09-11T07:44:39.357 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-11T07:44:39.367 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-11T07:44:39.378 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-11T07:44:39.379 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi110.front.sepia.ceph.com,smithi139.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-11T07:48:28.301 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi110.front.sepia.ceph.com'), Remote(name='ubuntu@smithi139.front.sepia.ceph.com')] 2024-09-11T07:48:28.302 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi110.front.sepia.ceph.com' 2024-09-11T07:48:28.304 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T07:48:28.384 DEBUG:teuthology.orchestra.run.smithi110:> true 2024-09-11T07:48:28.472 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi110.front.sepia.ceph.com' 2024-09-11T07:48:28.473 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi139.front.sepia.ceph.com' 2024-09-11T07:48:28.473 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T07:48:28.550 DEBUG:teuthology.orchestra.run.smithi139:> true 2024-09-11T07:48:28.623 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi139.front.sepia.ceph.com' 2024-09-11T07:48:28.623 INFO:teuthology.run_tasks:Running task clock... 2024-09-11T07:48:28.633 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-11T07:48:28.634 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-11T07:48:28.634 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T07:48:28.637 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-11T07:48:28.637 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T07:48:28.664 INFO:teuthology.orchestra.run.smithi110.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-11T07:48:28.679 INFO:teuthology.orchestra.run.smithi110.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-11T07:48:28.691 INFO:teuthology.orchestra.run.smithi139.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-11T07:48:28.703 INFO:teuthology.orchestra.run.smithi110.stderr:sudo: ntpd: command not found 2024-09-11T07:48:28.705 INFO:teuthology.orchestra.run.smithi139.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-11T07:48:28.715 INFO:teuthology.orchestra.run.smithi110.stdout:506 Cannot talk to daemon 2024-09-11T07:48:28.729 INFO:teuthology.orchestra.run.smithi110.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-11T07:48:28.729 INFO:teuthology.orchestra.run.smithi139.stderr:sudo: ntpd: command not found 2024-09-11T07:48:28.740 INFO:teuthology.orchestra.run.smithi139.stdout:506 Cannot talk to daemon 2024-09-11T07:48:28.743 INFO:teuthology.orchestra.run.smithi110.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-11T07:48:28.754 INFO:teuthology.orchestra.run.smithi139.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-11T07:48:28.768 INFO:teuthology.orchestra.run.smithi139.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-11T07:48:28.794 INFO:teuthology.orchestra.run.smithi110.stderr:bash: line 1: ntpq: command not found 2024-09-11T07:48:28.798 INFO:teuthology.orchestra.run.smithi110.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T07:48:28.798 INFO:teuthology.orchestra.run.smithi110.stdout:=============================================================================== 2024-09-11T07:48:28.798 INFO:teuthology.orchestra.run.smithi110.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T07:48:28.798 INFO:teuthology.orchestra.run.smithi110.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T07:48:28.798 INFO:teuthology.orchestra.run.smithi110.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T07:48:28.798 INFO:teuthology.orchestra.run.smithi110.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T07:48:28.826 INFO:teuthology.orchestra.run.smithi139.stderr:bash: line 1: ntpq: command not found 2024-09-11T07:48:28.830 INFO:teuthology.orchestra.run.smithi139.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T07:48:28.830 INFO:teuthology.orchestra.run.smithi139.stdout:=============================================================================== 2024-09-11T07:48:28.830 INFO:teuthology.orchestra.run.smithi139.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T07:48:28.830 INFO:teuthology.orchestra.run.smithi139.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T07:48:28.830 INFO:teuthology.orchestra.run.smithi139.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T07:48:28.830 INFO:teuthology.orchestra.run.smithi139.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T07:48:28.830 INFO:teuthology.run_tasks:Running task pexec... 2024-09-11T07:48:28.839 INFO:teuthology.task.pexec:Executing custom commands... 2024-09-11T07:48:28.840 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi110.front.sepia.ceph.com 2024-09-11T07:48:28.840 DEBUG:teuthology.orchestra.run.smithi110:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-11T07:48:28.841 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi139.front.sepia.ceph.com 2024-09-11T07:48:28.841 DEBUG:teuthology.orchestra.run.smithi139:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-11T07:48:29.191 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-09-11T07:48:29.209 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-11T07:48:29.209 INFO:teuthology.orchestra.run.smithi110.stdout: Package Architecture Version Repository Size 2024-09-11T07:48:29.210 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-11T07:48:29.210 INFO:teuthology.orchestra.run.smithi110.stdout:Removing: 2024-09-11T07:48:29.210 INFO:teuthology.orchestra.run.smithi110.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-11T07:48:29.210 INFO:teuthology.orchestra.run.smithi110.stdout:Removing unused dependencies: 2024-09-11T07:48:29.210 INFO:teuthology.orchestra.run.smithi110.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-11T07:48:29.210 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T07:48:29.210 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2024-09-11T07:48:29.210 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-11T07:48:29.210 INFO:teuthology.orchestra.run.smithi110.stdout:Remove 2 Packages 2024-09-11T07:48:29.210 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T07:48:29.211 INFO:teuthology.orchestra.run.smithi110.stdout:Freed space: 5.5 M 2024-09-11T07:48:29.211 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2024-09-11T07:48:29.214 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2024-09-11T07:48:29.214 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2024-09-11T07:48:29.218 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-09-11T07:48:29.230 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-11T07:48:29.230 INFO:teuthology.orchestra.run.smithi139.stdout: Package Architecture Version Repository Size 2024-09-11T07:48:29.231 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-11T07:48:29.231 INFO:teuthology.orchestra.run.smithi139.stdout:Removing: 2024-09-11T07:48:29.231 INFO:teuthology.orchestra.run.smithi139.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-11T07:48:29.231 INFO:teuthology.orchestra.run.smithi139.stdout:Removing unused dependencies: 2024-09-11T07:48:29.231 INFO:teuthology.orchestra.run.smithi139.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-11T07:48:29.231 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-11T07:48:29.231 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction Summary 2024-09-11T07:48:29.231 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-11T07:48:29.231 INFO:teuthology.orchestra.run.smithi139.stdout:Remove 2 Packages 2024-09-11T07:48:29.231 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-11T07:48:29.232 INFO:teuthology.orchestra.run.smithi139.stdout:Freed space: 5.5 M 2024-09-11T07:48:29.232 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction check 2024-09-11T07:48:29.235 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction check succeeded. 2024-09-11T07:48:29.235 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction test 2024-09-11T07:48:29.282 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2024-09-11T07:48:29.282 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2024-09-11T07:48:29.301 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction test succeeded. 2024-09-11T07:48:29.301 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction 2024-09-11T07:48:29.428 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2024-09-11T07:48:29.451 INFO:teuthology.orchestra.run.smithi139.stdout: Preparing : 1/1 2024-09-11T07:48:29.516 INFO:teuthology.orchestra.run.smithi110.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-11T07:48:29.527 INFO:teuthology.orchestra.run.smithi139.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-11T07:48:29.592 INFO:teuthology.orchestra.run.smithi110.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-11T07:48:29.593 INFO:teuthology.orchestra.run.smithi139.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-11T07:48:30.022 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-11T07:48:30.022 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-11T07:48:30.062 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-11T07:48:30.062 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-11T07:48:30.255 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-11T07:48:30.255 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-11T07:48:30.255 INFO:teuthology.orchestra.run.smithi139.stdout:Removed: 2024-09-11T07:48:30.255 INFO:teuthology.orchestra.run.smithi139.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-11T07:48:30.255 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-11T07:48:30.255 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-09-11T07:48:30.320 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-11T07:48:30.320 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T07:48:30.320 INFO:teuthology.orchestra.run.smithi110.stdout:Removed: 2024-09-11T07:48:30.320 INFO:teuthology.orchestra.run.smithi110.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-11T07:48:30.320 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T07:48:30.320 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-09-11T07:48:31.060 INFO:teuthology.orchestra.run.smithi139.stdout:Last metadata expiration check: 0:00:46 ago on Wed 11 Sep 2024 07:47:44 AM UTC. 2024-09-11T07:48:31.117 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:00:56 ago on Wed 11 Sep 2024 07:47:34 AM UTC. 2024-09-11T07:48:31.201 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-09-11T07:48:31.202 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-11T07:48:31.202 INFO:teuthology.orchestra.run.smithi139.stdout: Package Architecture Version Repository Size 2024-09-11T07:48:31.202 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-11T07:48:31.202 INFO:teuthology.orchestra.run.smithi139.stdout:Installing: 2024-09-11T07:48:31.202 INFO:teuthology.orchestra.run.smithi139.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-11T07:48:31.202 INFO:teuthology.orchestra.run.smithi139.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-11T07:48:31.202 INFO:teuthology.orchestra.run.smithi139.stdout:Installing dependencies: 2024-09-11T07:48:31.202 INFO:teuthology.orchestra.run.smithi139.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-11T07:48:31.202 INFO:teuthology.orchestra.run.smithi139.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-11T07:48:31.203 INFO:teuthology.orchestra.run.smithi139.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-11T07:48:31.203 INFO:teuthology.orchestra.run.smithi139.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-11T07:48:31.203 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-11T07:48:31.203 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction Summary 2024-09-11T07:48:31.203 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-11T07:48:31.203 INFO:teuthology.orchestra.run.smithi139.stdout:Install 6 Packages 2024-09-11T07:48:31.203 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-11T07:48:31.217 INFO:teuthology.orchestra.run.smithi139.stdout:Total download size: 2.0 M 2024-09-11T07:48:31.217 INFO:teuthology.orchestra.run.smithi139.stdout:Installed size: 9.2 M 2024-09-11T07:48:31.217 INFO:teuthology.orchestra.run.smithi139.stdout:Downloading Packages: 2024-09-11T07:48:31.253 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-09-11T07:48:31.254 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-11T07:48:31.254 INFO:teuthology.orchestra.run.smithi110.stdout: Package Architecture Version Repository Size 2024-09-11T07:48:31.254 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-11T07:48:31.254 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2024-09-11T07:48:31.254 INFO:teuthology.orchestra.run.smithi110.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-11T07:48:31.254 INFO:teuthology.orchestra.run.smithi110.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-11T07:48:31.254 INFO:teuthology.orchestra.run.smithi110.stdout:Installing dependencies: 2024-09-11T07:48:31.254 INFO:teuthology.orchestra.run.smithi110.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-11T07:48:31.254 INFO:teuthology.orchestra.run.smithi110.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-11T07:48:31.255 INFO:teuthology.orchestra.run.smithi110.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-11T07:48:31.255 INFO:teuthology.orchestra.run.smithi110.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-11T07:48:31.255 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T07:48:31.255 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2024-09-11T07:48:31.255 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-11T07:48:31.255 INFO:teuthology.orchestra.run.smithi110.stdout:Install 6 Packages 2024-09-11T07:48:31.255 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T07:48:31.265 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 2.0 M 2024-09-11T07:48:31.265 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 9.2 M 2024-09-11T07:48:31.265 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2024-09-11T07:48:31.368 INFO:teuthology.orchestra.run.smithi139.stdout:(1/6): nvmetcli-0.7-3.el9.noarch.rpm 420 kB/s | 44 kB 00:00 2024-09-11T07:48:31.393 INFO:teuthology.orchestra.run.smithi139.stdout:(2/6): libnvme-1.9-3.el9.x86_64.rpm 808 kB/s | 106 kB 00:00 2024-09-11T07:48:31.460 INFO:teuthology.orchestra.run.smithi139.stdout:(3/6): nvme-cli-2.9.1-6.el9.x86_64.rpm 4.5 MB/s | 906 kB 00:00 2024-09-11T07:48:31.477 INFO:teuthology.orchestra.run.smithi139.stdout:(4/6): python3-configshell-1.1.30-1.el9.noarch. 657 kB/s | 72 kB 00:00 2024-09-11T07:48:31.502 INFO:teuthology.orchestra.run.smithi139.stdout:(5/6): python3-kmod-0.9-32.el9.x86_64.rpm 772 kB/s | 84 kB 00:00 2024-09-11T07:48:31.544 INFO:teuthology.orchestra.run.smithi139.stdout:(6/6): python3-urwid-2.1.2-4.el9.x86_64.rpm 9.8 MB/s | 837 kB 00:00 2024-09-11T07:48:31.544 INFO:teuthology.orchestra.run.smithi139.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:48:31.544 INFO:teuthology.orchestra.run.smithi139.stdout:Total 6.1 MB/s | 2.0 MB 00:00 2024-09-11T07:48:31.618 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction check 2024-09-11T07:48:31.632 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction check succeeded. 2024-09-11T07:48:31.632 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction test 2024-09-11T07:48:31.656 INFO:teuthology.orchestra.run.smithi110.stdout:(1/6): nvmetcli-0.7-3.el9.noarch.rpm 316 kB/s | 44 kB 00:00 2024-09-11T07:48:31.682 INFO:teuthology.orchestra.run.smithi110.stdout:(2/6): libnvme-1.9-3.el9.x86_64.rpm 639 kB/s | 106 kB 00:00 2024-09-11T07:48:31.732 INFO:teuthology.orchestra.run.smithi110.stdout:(3/6): python3-kmod-0.9-32.el9.x86_64.rpm 1.6 MB/s | 84 kB 00:00 2024-09-11T07:48:31.757 INFO:teuthology.orchestra.run.smithi110.stdout:(4/6): python3-configshell-1.1.30-1.el9.noarch. 712 kB/s | 72 kB 00:00 2024-09-11T07:48:31.825 INFO:teuthology.orchestra.run.smithi110.stdout:(5/6): nvme-cli-2.9.1-6.el9.x86_64.rpm 2.9 MB/s | 906 kB 00:00 2024-09-11T07:48:31.916 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction test succeeded. 2024-09-11T07:48:31.916 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction 2024-09-11T07:48:31.925 INFO:teuthology.orchestra.run.smithi110.stdout:(6/6): python3-urwid-2.1.2-4.el9.x86_64.rpm 4.3 MB/s | 837 kB 00:00 2024-09-11T07:48:31.925 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:48:31.925 INFO:teuthology.orchestra.run.smithi110.stdout:Total 3.0 MB/s | 2.0 MB 00:00 2024-09-11T07:48:31.999 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2024-09-11T07:48:32.012 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2024-09-11T07:48:32.012 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2024-09-11T07:48:32.325 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2024-09-11T07:48:32.325 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2024-09-11T07:48:32.344 INFO:teuthology.orchestra.run.smithi139.stdout: Preparing : 1/1 2024-09-11T07:48:32.441 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/6 2024-09-11T07:48:32.473 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/6 2024-09-11T07:48:32.501 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/6 2024-09-11T07:48:32.707 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/6 2024-09-11T07:48:32.740 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/6 2024-09-11T07:48:32.762 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2024-09-11T07:48:32.853 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/6 2024-09-11T07:48:32.896 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/6 2024-09-11T07:48:32.933 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/6 2024-09-11T07:48:33.148 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/6 2024-09-11T07:48:33.164 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/6 2024-09-11T07:48:33.189 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/6 2024-09-11T07:48:33.194 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/6 2024-09-11T07:48:33.630 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/6 2024-09-11T07:48:33.669 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/6 2024-09-11T07:48:34.137 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/6 2024-09-11T07:48:34.137 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/6 2024-09-11T07:48:34.137 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/6 2024-09-11T07:48:34.137 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/6 2024-09-11T07:48:34.137 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/6 2024-09-11T07:48:34.137 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/6 2024-09-11T07:48:34.504 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/6 2024-09-11T07:48:34.504 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-11T07:48:34.504 INFO:teuthology.orchestra.run.smithi139.stdout:Installed: 2024-09-11T07:48:34.504 INFO:teuthology.orchestra.run.smithi139.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-11T07:48:34.504 INFO:teuthology.orchestra.run.smithi139.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-11T07:48:34.504 INFO:teuthology.orchestra.run.smithi139.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-11T07:48:34.505 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-11T07:48:34.505 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-09-11T07:48:34.620 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/6 2024-09-11T07:48:34.620 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/6 2024-09-11T07:48:34.620 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/6 2024-09-11T07:48:34.620 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/6 2024-09-11T07:48:34.620 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/6 2024-09-11T07:48:34.620 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/6 2024-09-11T07:48:34.870 DEBUG:teuthology.parallel:result is None 2024-09-11T07:48:35.019 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/6 2024-09-11T07:48:35.019 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T07:48:35.019 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2024-09-11T07:48:35.019 INFO:teuthology.orchestra.run.smithi110.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-11T07:48:35.019 INFO:teuthology.orchestra.run.smithi110.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-11T07:48:35.019 INFO:teuthology.orchestra.run.smithi110.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-11T07:48:35.019 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T07:48:35.019 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-09-11T07:48:35.339 DEBUG:teuthology.parallel:result is None 2024-09-11T07:48:35.340 INFO:teuthology.run_tasks:Running task nvme_loop... 2024-09-11T07:48:35.353 INFO:tasks.nvme_loop:Setting up nvme_loop on scratch devices... 2024-09-11T07:48:35.353 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-11T07:48:35.353 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/scratch_devs of=/dev/stdout 2024-09-11T07:48:35.370 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-11T07:48:35.370 DEBUG:teuthology.orchestra.run.smithi110:> stat /dev/vg_nvme/lv_1 2024-09-11T07:48:35.426 INFO:teuthology.orchestra.run.smithi110.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-11T07:48:35.426 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T07:48:35.426 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 5h/5d Inode: 961 Links: 1 2024-09-11T07:48:35.426 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T07:48:35.426 INFO:teuthology.orchestra.run.smithi110.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T07:48:35.426 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 2024-09-11 07:48:33.981660271 +0000 2024-09-11T07:48:35.426 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2024-09-11 07:48:33.730663005 +0000 2024-09-11T07:48:35.426 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2024-09-11 07:48:33.730663005 +0000 2024-09-11T07:48:35.427 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: 2024-09-11 07:48:33.730663005 +0000 2024-09-11T07:48:35.427 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-11T07:48:35.491 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2024-09-11T07:48:35.492 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2024-09-11T07:48:35.492 INFO:teuthology.orchestra.run.smithi110.stderr:512 bytes copied, 0.000111752 s, 4.6 MB/s 2024-09-11T07:48:35.493 DEBUG:teuthology.orchestra.run.smithi110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-11T07:48:35.549 DEBUG:teuthology.orchestra.run.smithi110:> stat /dev/vg_nvme/lv_2 2024-09-11T07:48:35.604 INFO:teuthology.orchestra.run.smithi110.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-11T07:48:35.604 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T07:48:35.605 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 5h/5d Inode: 984 Links: 1 2024-09-11T07:48:35.605 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T07:48:35.605 INFO:teuthology.orchestra.run.smithi110.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T07:48:35.605 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 2024-09-11 07:48:33.981660271 +0000 2024-09-11T07:48:35.605 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2024-09-11 07:48:33.733662973 +0000 2024-09-11T07:48:35.605 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2024-09-11 07:48:33.733662973 +0000 2024-09-11T07:48:35.605 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: 2024-09-11 07:48:33.733662973 +0000 2024-09-11T07:48:35.605 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-11T07:48:35.667 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2024-09-11T07:48:35.667 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2024-09-11T07:48:35.667 INFO:teuthology.orchestra.run.smithi110.stderr:512 bytes copied, 8.3759e-05 s, 6.1 MB/s 2024-09-11T07:48:35.668 DEBUG:teuthology.orchestra.run.smithi110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-11T07:48:35.724 DEBUG:teuthology.orchestra.run.smithi110:> stat /dev/vg_nvme/lv_3 2024-09-11T07:48:35.780 INFO:teuthology.orchestra.run.smithi110.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-11T07:48:35.780 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T07:48:35.780 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 5h/5d Inode: 968 Links: 1 2024-09-11T07:48:35.780 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T07:48:35.780 INFO:teuthology.orchestra.run.smithi110.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T07:48:35.781 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 2024-09-11 07:48:33.982660260 +0000 2024-09-11T07:48:35.781 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2024-09-11 07:48:33.731662994 +0000 2024-09-11T07:48:35.781 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2024-09-11 07:48:33.731662994 +0000 2024-09-11T07:48:35.781 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: 2024-09-11 07:48:33.731662994 +0000 2024-09-11T07:48:35.781 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-11T07:48:35.868 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2024-09-11T07:48:35.868 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2024-09-11T07:48:35.868 INFO:teuthology.orchestra.run.smithi110.stderr:512 bytes copied, 0.000102088 s, 5.0 MB/s 2024-09-11T07:48:35.869 DEBUG:teuthology.orchestra.run.smithi110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-11T07:48:35.885 DEBUG:teuthology.orchestra.run.smithi110:> stat /dev/vg_nvme/lv_4 2024-09-11T07:48:35.940 INFO:teuthology.orchestra.run.smithi110.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-11T07:48:35.940 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T07:48:35.940 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 5h/5d Inode: 972 Links: 1 2024-09-11T07:48:35.940 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T07:48:35.940 INFO:teuthology.orchestra.run.smithi110.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T07:48:35.940 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 2024-09-11 07:48:33.982660260 +0000 2024-09-11T07:48:35.940 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2024-09-11 07:48:33.731662994 +0000 2024-09-11T07:48:35.940 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2024-09-11 07:48:33.731662994 +0000 2024-09-11T07:48:35.940 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: 2024-09-11 07:48:33.731662994 +0000 2024-09-11T07:48:35.941 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-11T07:48:36.003 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2024-09-11T07:48:36.003 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2024-09-11T07:48:36.003 INFO:teuthology.orchestra.run.smithi110.stderr:512 bytes copied, 0.000102377 s, 5.0 MB/s 2024-09-11T07:48:36.005 DEBUG:teuthology.orchestra.run.smithi110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-11T07:48:36.061 DEBUG:teuthology.orchestra.run.smithi110:> sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype 2024-09-11T07:48:36.198 INFO:teuthology.orchestra.run.smithi110.stdout:loop 2024-09-11T07:48:36.200 INFO:tasks.nvme_loop:Connecting nvme_loop smithi110:/dev/vg_nvme/lv_1... 2024-09-11T07:48:36.200 DEBUG:teuthology.orchestra.run.smithi110:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1 && echo /dev/vg_nvme/lv_1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_1 /sys/kernel/config/nvmet/ports/1/subsystems/lv_1 && sudo nvme connect -t loop -n lv_1 -q hostnqn 2024-09-11T07:48:36.233 INFO:teuthology.orchestra.run.smithi110.stdout:1 2024-09-11T07:48:36.252 INFO:teuthology.orchestra.run.smithi110.stdout:/dev/vg_nvme/lv_1 2024-09-11T07:48:36.273 INFO:teuthology.orchestra.run.smithi110.stdout:1 2024-09-11T07:48:36.316 INFO:teuthology.orchestra.run.smithi110.stdout:connecting to device: nvme1 2024-09-11T07:48:36.319 INFO:tasks.nvme_loop:Connecting nvme_loop smithi110:/dev/vg_nvme/lv_2... 2024-09-11T07:48:36.319 DEBUG:teuthology.orchestra.run.smithi110:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1 && echo /dev/vg_nvme/lv_2 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_2 /sys/kernel/config/nvmet/ports/1/subsystems/lv_2 && sudo nvme connect -t loop -n lv_2 -q hostnqn 2024-09-11T07:48:36.390 INFO:teuthology.orchestra.run.smithi110.stdout:1 2024-09-11T07:48:36.411 INFO:teuthology.orchestra.run.smithi110.stdout:/dev/vg_nvme/lv_2 2024-09-11T07:48:36.421 INFO:teuthology.orchestra.run.smithi110.stdout:1 2024-09-11T07:48:36.451 INFO:teuthology.orchestra.run.smithi110.stdout:connecting to device: nvme2 2024-09-11T07:48:36.453 INFO:tasks.nvme_loop:Connecting nvme_loop smithi110:/dev/vg_nvme/lv_3... 2024-09-11T07:48:36.454 DEBUG:teuthology.orchestra.run.smithi110:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1 && echo /dev/vg_nvme/lv_3 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_3 /sys/kernel/config/nvmet/ports/1/subsystems/lv_3 && sudo nvme connect -t loop -n lv_3 -q hostnqn 2024-09-11T07:48:36.528 INFO:teuthology.orchestra.run.smithi110.stdout:1 2024-09-11T07:48:36.549 INFO:teuthology.orchestra.run.smithi110.stdout:/dev/vg_nvme/lv_3 2024-09-11T07:48:36.559 INFO:teuthology.orchestra.run.smithi110.stdout:1 2024-09-11T07:48:36.590 INFO:teuthology.orchestra.run.smithi110.stdout:connecting to device: nvme3 2024-09-11T07:48:36.592 INFO:tasks.nvme_loop:Connecting nvme_loop smithi110:/dev/vg_nvme/lv_4... 2024-09-11T07:48:36.592 DEBUG:teuthology.orchestra.run.smithi110:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1 && echo /dev/vg_nvme/lv_4 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_4 /sys/kernel/config/nvmet/ports/1/subsystems/lv_4 && sudo nvme connect -t loop -n lv_4 -q hostnqn 2024-09-11T07:48:36.666 INFO:teuthology.orchestra.run.smithi110.stdout:1 2024-09-11T07:48:36.686 INFO:teuthology.orchestra.run.smithi110.stdout:/dev/vg_nvme/lv_4 2024-09-11T07:48:36.696 INFO:teuthology.orchestra.run.smithi110.stdout:1 2024-09-11T07:48:36.732 INFO:teuthology.orchestra.run.smithi110.stdout:connecting to device: nvme4 2024-09-11T07:48:36.737 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-11T07:48:36.737 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/scratch_devs of=/dev/stdout 2024-09-11T07:48:36.792 DEBUG:teuthology.orchestra.run.smithi110:> sudo nvme list -o json 2024-09-11T07:48:36.858 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-11T07:48:36.858 INFO:teuthology.orchestra.run.smithi110.stdout: "Devices":[ 2024-09-11T07:48:36.858 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-11T07:48:36.858 INFO:teuthology.orchestra.run.smithi110.stdout: "NameSpace":1, 2024-09-11T07:48:36.858 INFO:teuthology.orchestra.run.smithi110.stdout: "DevicePath":"/dev/nvme4n1", 2024-09-11T07:48:36.858 INFO:teuthology.orchestra.run.smithi110.stdout: "GenericPath":"/dev/ng4n1", 2024-09-11T07:48:36.858 INFO:teuthology.orchestra.run.smithi110.stdout: "Firmware":"5.14.0-5", 2024-09-11T07:48:36.858 INFO:teuthology.orchestra.run.smithi110.stdout: "ModelNumber":"Linux", 2024-09-11T07:48:36.859 INFO:teuthology.orchestra.run.smithi110.stdout: "SerialNumber":"5686d6b55281be7d01c3", 2024-09-11T07:48:36.859 INFO:teuthology.orchestra.run.smithi110.stdout: "UsedBytes":95995035648, 2024-09-11T07:48:36.859 INFO:teuthology.orchestra.run.smithi110.stdout: "MaximumLBA":187490304, 2024-09-11T07:48:36.859 INFO:teuthology.orchestra.run.smithi110.stdout: "PhysicalSize":95995035648, 2024-09-11T07:48:36.859 INFO:teuthology.orchestra.run.smithi110.stdout: "SectorSize":512 2024-09-11T07:48:36.859 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T07:48:36.859 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-11T07:48:36.859 INFO:teuthology.orchestra.run.smithi110.stdout: "NameSpace":1, 2024-09-11T07:48:36.859 INFO:teuthology.orchestra.run.smithi110.stdout: "DevicePath":"/dev/nvme3n1", 2024-09-11T07:48:36.859 INFO:teuthology.orchestra.run.smithi110.stdout: "GenericPath":"/dev/ng3n1", 2024-09-11T07:48:36.859 INFO:teuthology.orchestra.run.smithi110.stdout: "Firmware":"5.14.0-5", 2024-09-11T07:48:36.859 INFO:teuthology.orchestra.run.smithi110.stdout: "ModelNumber":"Linux", 2024-09-11T07:48:36.859 INFO:teuthology.orchestra.run.smithi110.stdout: "SerialNumber":"de6f9c3c1337e1563f8b", 2024-09-11T07:48:36.860 INFO:teuthology.orchestra.run.smithi110.stdout: "UsedBytes":95995035648, 2024-09-11T07:48:36.860 INFO:teuthology.orchestra.run.smithi110.stdout: "MaximumLBA":187490304, 2024-09-11T07:48:36.860 INFO:teuthology.orchestra.run.smithi110.stdout: "PhysicalSize":95995035648, 2024-09-11T07:48:36.860 INFO:teuthology.orchestra.run.smithi110.stdout: "SectorSize":512 2024-09-11T07:48:36.860 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T07:48:36.860 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-11T07:48:36.860 INFO:teuthology.orchestra.run.smithi110.stdout: "NameSpace":1, 2024-09-11T07:48:36.860 INFO:teuthology.orchestra.run.smithi110.stdout: "DevicePath":"/dev/nvme2n1", 2024-09-11T07:48:36.860 INFO:teuthology.orchestra.run.smithi110.stdout: "GenericPath":"/dev/ng2n1", 2024-09-11T07:48:36.860 INFO:teuthology.orchestra.run.smithi110.stdout: "Firmware":"5.14.0-5", 2024-09-11T07:48:36.860 INFO:teuthology.orchestra.run.smithi110.stdout: "ModelNumber":"Linux", 2024-09-11T07:48:36.860 INFO:teuthology.orchestra.run.smithi110.stdout: "SerialNumber":"be3bbd0bcdc85b0f8e9e", 2024-09-11T07:48:36.860 INFO:teuthology.orchestra.run.smithi110.stdout: "UsedBytes":95995035648, 2024-09-11T07:48:36.861 INFO:teuthology.orchestra.run.smithi110.stdout: "MaximumLBA":187490304, 2024-09-11T07:48:36.861 INFO:teuthology.orchestra.run.smithi110.stdout: "PhysicalSize":95995035648, 2024-09-11T07:48:36.861 INFO:teuthology.orchestra.run.smithi110.stdout: "SectorSize":512 2024-09-11T07:48:36.861 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T07:48:36.861 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-11T07:48:36.861 INFO:teuthology.orchestra.run.smithi110.stdout: "NameSpace":1, 2024-09-11T07:48:36.861 INFO:teuthology.orchestra.run.smithi110.stdout: "DevicePath":"/dev/nvme1n1", 2024-09-11T07:48:36.861 INFO:teuthology.orchestra.run.smithi110.stdout: "GenericPath":"/dev/ng1n1", 2024-09-11T07:48:36.861 INFO:teuthology.orchestra.run.smithi110.stdout: "Firmware":"5.14.0-5", 2024-09-11T07:48:36.861 INFO:teuthology.orchestra.run.smithi110.stdout: "ModelNumber":"Linux", 2024-09-11T07:48:36.861 INFO:teuthology.orchestra.run.smithi110.stdout: "SerialNumber":"6156f70d4dbbca17f65d", 2024-09-11T07:48:36.861 INFO:teuthology.orchestra.run.smithi110.stdout: "UsedBytes":95995035648, 2024-09-11T07:48:36.862 INFO:teuthology.orchestra.run.smithi110.stdout: "MaximumLBA":187490304, 2024-09-11T07:48:36.862 INFO:teuthology.orchestra.run.smithi110.stdout: "PhysicalSize":95995035648, 2024-09-11T07:48:36.862 INFO:teuthology.orchestra.run.smithi110.stdout: "SectorSize":512 2024-09-11T07:48:36.862 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-11T07:48:36.862 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-11T07:48:36.862 INFO:teuthology.orchestra.run.smithi110.stdout: "NameSpace":1, 2024-09-11T07:48:36.862 INFO:teuthology.orchestra.run.smithi110.stdout: "DevicePath":"/dev/nvme0n1", 2024-09-11T07:48:36.862 INFO:teuthology.orchestra.run.smithi110.stdout: "GenericPath":"/dev/ng0n1", 2024-09-11T07:48:36.862 INFO:teuthology.orchestra.run.smithi110.stdout: "Firmware":"E2010325", 2024-09-11T07:48:36.862 INFO:teuthology.orchestra.run.smithi110.stdout: "ModelNumber":"INTEL SSDPED1D480GA", 2024-09-11T07:48:36.862 INFO:teuthology.orchestra.run.smithi110.stdout: "SerialNumber":"PHMB7476002T480DGN", 2024-09-11T07:48:36.862 INFO:teuthology.orchestra.run.smithi110.stdout: "UsedBytes":480103981056, 2024-09-11T07:48:36.862 INFO:teuthology.orchestra.run.smithi110.stdout: "MaximumLBA":937703088, 2024-09-11T07:48:36.862 INFO:teuthology.orchestra.run.smithi110.stdout: "PhysicalSize":480103981056, 2024-09-11T07:48:36.862 INFO:teuthology.orchestra.run.smithi110.stdout: "SectorSize":512 2024-09-11T07:48:36.862 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-11T07:48:36.863 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-11T07:48:36.863 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-11T07:48:36.863 INFO:tasks.nvme_loop:new_devs ['/dev/nvme4n1', '/dev/nvme3n1', '/dev/nvme2n1', '/dev/nvme1n1'] 2024-09-11T07:48:36.864 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-11T07:48:36.864 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/scratch_devs 2024-09-11T07:48:36.929 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-11T07:48:36.929 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/scratch_devs of=/dev/stdout 2024-09-11T07:48:36.949 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-11T07:48:36.949 DEBUG:teuthology.orchestra.run.smithi139:> stat /dev/vg_nvme/lv_1 2024-09-11T07:48:37.012 INFO:teuthology.orchestra.run.smithi139.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-11T07:48:37.012 INFO:teuthology.orchestra.run.smithi139.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T07:48:37.012 INFO:teuthology.orchestra.run.smithi139.stdout:Device: 5h/5d Inode: 981 Links: 1 2024-09-11T07:48:37.012 INFO:teuthology.orchestra.run.smithi139.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T07:48:37.012 INFO:teuthology.orchestra.run.smithi139.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T07:48:37.012 INFO:teuthology.orchestra.run.smithi139.stdout:Access: 2024-09-11 07:48:33.530689956 +0000 2024-09-11T07:48:37.012 INFO:teuthology.orchestra.run.smithi139.stdout:Modify: 2024-09-11 07:48:33.272699382 +0000 2024-09-11T07:48:37.012 INFO:teuthology.orchestra.run.smithi139.stdout:Change: 2024-09-11 07:48:33.272699382 +0000 2024-09-11T07:48:37.013 INFO:teuthology.orchestra.run.smithi139.stdout: Birth: 2024-09-11 07:48:33.272699382 +0000 2024-09-11T07:48:37.013 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-11T07:48:37.081 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records in 2024-09-11T07:48:37.082 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records out 2024-09-11T07:48:37.082 INFO:teuthology.orchestra.run.smithi139.stderr:512 bytes copied, 0.000273402 s, 1.9 MB/s 2024-09-11T07:48:37.083 DEBUG:teuthology.orchestra.run.smithi139:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-11T07:48:37.141 DEBUG:teuthology.orchestra.run.smithi139:> stat /dev/vg_nvme/lv_2 2024-09-11T07:48:37.198 INFO:teuthology.orchestra.run.smithi139.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-11T07:48:37.198 INFO:teuthology.orchestra.run.smithi139.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T07:48:37.198 INFO:teuthology.orchestra.run.smithi139.stdout:Device: 5h/5d Inode: 979 Links: 1 2024-09-11T07:48:37.198 INFO:teuthology.orchestra.run.smithi139.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T07:48:37.198 INFO:teuthology.orchestra.run.smithi139.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T07:48:37.198 INFO:teuthology.orchestra.run.smithi139.stdout:Access: 2024-09-11 07:48:33.530689956 +0000 2024-09-11T07:48:37.199 INFO:teuthology.orchestra.run.smithi139.stdout:Modify: 2024-09-11 07:48:33.272699382 +0000 2024-09-11T07:48:37.199 INFO:teuthology.orchestra.run.smithi139.stdout:Change: 2024-09-11 07:48:33.272699382 +0000 2024-09-11T07:48:37.199 INFO:teuthology.orchestra.run.smithi139.stdout: Birth: 2024-09-11 07:48:33.272699382 +0000 2024-09-11T07:48:37.199 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-11T07:48:37.263 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records in 2024-09-11T07:48:37.263 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records out 2024-09-11T07:48:37.263 INFO:teuthology.orchestra.run.smithi139.stderr:512 bytes copied, 0.00022852 s, 2.2 MB/s 2024-09-11T07:48:37.265 DEBUG:teuthology.orchestra.run.smithi139:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-11T07:48:37.322 DEBUG:teuthology.orchestra.run.smithi139:> stat /dev/vg_nvme/lv_3 2024-09-11T07:48:37.378 INFO:teuthology.orchestra.run.smithi139.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-11T07:48:37.378 INFO:teuthology.orchestra.run.smithi139.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T07:48:37.378 INFO:teuthology.orchestra.run.smithi139.stdout:Device: 5h/5d Inode: 972 Links: 1 2024-09-11T07:48:37.378 INFO:teuthology.orchestra.run.smithi139.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T07:48:37.378 INFO:teuthology.orchestra.run.smithi139.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T07:48:37.378 INFO:teuthology.orchestra.run.smithi139.stdout:Access: 2024-09-11 07:48:33.530689956 +0000 2024-09-11T07:48:37.379 INFO:teuthology.orchestra.run.smithi139.stdout:Modify: 2024-09-11 07:48:33.271699403 +0000 2024-09-11T07:48:37.379 INFO:teuthology.orchestra.run.smithi139.stdout:Change: 2024-09-11 07:48:33.271699403 +0000 2024-09-11T07:48:37.379 INFO:teuthology.orchestra.run.smithi139.stdout: Birth: 2024-09-11 07:48:33.271699403 +0000 2024-09-11T07:48:37.379 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-11T07:48:37.441 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records in 2024-09-11T07:48:37.442 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records out 2024-09-11T07:48:37.442 INFO:teuthology.orchestra.run.smithi139.stderr:512 bytes copied, 0.00017173 s, 3.0 MB/s 2024-09-11T07:48:37.443 DEBUG:teuthology.orchestra.run.smithi139:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-11T07:48:37.499 DEBUG:teuthology.orchestra.run.smithi139:> stat /dev/vg_nvme/lv_4 2024-09-11T07:48:37.555 INFO:teuthology.orchestra.run.smithi139.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-11T07:48:37.555 INFO:teuthology.orchestra.run.smithi139.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T07:48:37.555 INFO:teuthology.orchestra.run.smithi139.stdout:Device: 5h/5d Inode: 971 Links: 1 2024-09-11T07:48:37.555 INFO:teuthology.orchestra.run.smithi139.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T07:48:37.555 INFO:teuthology.orchestra.run.smithi139.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T07:48:37.555 INFO:teuthology.orchestra.run.smithi139.stdout:Access: 2024-09-11 07:48:33.531689826 +0000 2024-09-11T07:48:37.555 INFO:teuthology.orchestra.run.smithi139.stdout:Modify: 2024-09-11 07:48:33.271699403 +0000 2024-09-11T07:48:37.555 INFO:teuthology.orchestra.run.smithi139.stdout:Change: 2024-09-11 07:48:33.271699403 +0000 2024-09-11T07:48:37.555 INFO:teuthology.orchestra.run.smithi139.stdout: Birth: 2024-09-11 07:48:33.271699403 +0000 2024-09-11T07:48:37.556 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-11T07:48:37.618 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records in 2024-09-11T07:48:37.618 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records out 2024-09-11T07:48:37.618 INFO:teuthology.orchestra.run.smithi139.stderr:512 bytes copied, 0.000187263 s, 2.7 MB/s 2024-09-11T07:48:37.620 DEBUG:teuthology.orchestra.run.smithi139:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-11T07:48:37.676 DEBUG:teuthology.orchestra.run.smithi139:> sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype 2024-09-11T07:48:37.814 INFO:teuthology.orchestra.run.smithi139.stdout:loop 2024-09-11T07:48:37.816 INFO:tasks.nvme_loop:Connecting nvme_loop smithi139:/dev/vg_nvme/lv_1... 2024-09-11T07:48:37.816 DEBUG:teuthology.orchestra.run.smithi139:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1 && echo /dev/vg_nvme/lv_1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_1 /sys/kernel/config/nvmet/ports/1/subsystems/lv_1 && sudo nvme connect -t loop -n lv_1 -q hostnqn 2024-09-11T07:48:37.849 INFO:teuthology.orchestra.run.smithi139.stdout:1 2024-09-11T07:48:37.873 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/vg_nvme/lv_1 2024-09-11T07:48:37.882 INFO:teuthology.orchestra.run.smithi139.stdout:1 2024-09-11T07:48:37.930 INFO:teuthology.orchestra.run.smithi139.stdout:connecting to device: nvme1 2024-09-11T07:48:37.932 INFO:tasks.nvme_loop:Connecting nvme_loop smithi139:/dev/vg_nvme/lv_2... 2024-09-11T07:48:37.932 DEBUG:teuthology.orchestra.run.smithi139:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1 && echo /dev/vg_nvme/lv_2 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_2 /sys/kernel/config/nvmet/ports/1/subsystems/lv_2 && sudo nvme connect -t loop -n lv_2 -q hostnqn 2024-09-11T07:48:38.012 INFO:teuthology.orchestra.run.smithi139.stdout:1 2024-09-11T07:48:38.032 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/vg_nvme/lv_2 2024-09-11T07:48:38.043 INFO:teuthology.orchestra.run.smithi139.stdout:1 2024-09-11T07:48:38.074 INFO:teuthology.orchestra.run.smithi139.stdout:connecting to device: nvme2 2024-09-11T07:48:38.076 INFO:tasks.nvme_loop:Connecting nvme_loop smithi139:/dev/vg_nvme/lv_3... 2024-09-11T07:48:38.076 DEBUG:teuthology.orchestra.run.smithi139:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1 && echo /dev/vg_nvme/lv_3 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_3 /sys/kernel/config/nvmet/ports/1/subsystems/lv_3 && sudo nvme connect -t loop -n lv_3 -q hostnqn 2024-09-11T07:48:38.150 INFO:teuthology.orchestra.run.smithi139.stdout:1 2024-09-11T07:48:38.170 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/vg_nvme/lv_3 2024-09-11T07:48:38.181 INFO:teuthology.orchestra.run.smithi139.stdout:1 2024-09-11T07:48:38.216 INFO:teuthology.orchestra.run.smithi139.stdout:connecting to device: nvme3 2024-09-11T07:48:38.218 INFO:tasks.nvme_loop:Connecting nvme_loop smithi139:/dev/vg_nvme/lv_4... 2024-09-11T07:48:38.218 DEBUG:teuthology.orchestra.run.smithi139:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1 && echo /dev/vg_nvme/lv_4 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_4 /sys/kernel/config/nvmet/ports/1/subsystems/lv_4 && sudo nvme connect -t loop -n lv_4 -q hostnqn 2024-09-11T07:48:38.292 INFO:teuthology.orchestra.run.smithi139.stdout:1 2024-09-11T07:48:38.312 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/vg_nvme/lv_4 2024-09-11T07:48:38.322 INFO:teuthology.orchestra.run.smithi139.stdout:1 2024-09-11T07:48:38.353 INFO:teuthology.orchestra.run.smithi139.stdout:connecting to device: nvme4 2024-09-11T07:48:38.359 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-11T07:48:38.359 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/scratch_devs of=/dev/stdout 2024-09-11T07:48:38.415 DEBUG:teuthology.orchestra.run.smithi139:> sudo nvme list -o json 2024-09-11T07:48:38.481 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-11T07:48:38.481 INFO:teuthology.orchestra.run.smithi139.stdout: "Devices":[ 2024-09-11T07:48:38.481 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-11T07:48:38.481 INFO:teuthology.orchestra.run.smithi139.stdout: "NameSpace":1, 2024-09-11T07:48:38.482 INFO:teuthology.orchestra.run.smithi139.stdout: "DevicePath":"/dev/nvme4n1", 2024-09-11T07:48:38.482 INFO:teuthology.orchestra.run.smithi139.stdout: "GenericPath":"/dev/ng4n1", 2024-09-11T07:48:38.482 INFO:teuthology.orchestra.run.smithi139.stdout: "Firmware":"5.14.0-5", 2024-09-11T07:48:38.482 INFO:teuthology.orchestra.run.smithi139.stdout: "ModelNumber":"Linux", 2024-09-11T07:48:38.482 INFO:teuthology.orchestra.run.smithi139.stdout: "SerialNumber":"1c8c0cd73d601cca786f", 2024-09-11T07:48:38.482 INFO:teuthology.orchestra.run.smithi139.stdout: "UsedBytes":95995035648, 2024-09-11T07:48:38.482 INFO:teuthology.orchestra.run.smithi139.stdout: "MaximumLBA":187490304, 2024-09-11T07:48:38.482 INFO:teuthology.orchestra.run.smithi139.stdout: "PhysicalSize":95995035648, 2024-09-11T07:48:38.482 INFO:teuthology.orchestra.run.smithi139.stdout: "SectorSize":512 2024-09-11T07:48:38.482 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-11T07:48:38.482 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-11T07:48:38.482 INFO:teuthology.orchestra.run.smithi139.stdout: "NameSpace":1, 2024-09-11T07:48:38.482 INFO:teuthology.orchestra.run.smithi139.stdout: "DevicePath":"/dev/nvme3n1", 2024-09-11T07:48:38.483 INFO:teuthology.orchestra.run.smithi139.stdout: "GenericPath":"/dev/ng3n1", 2024-09-11T07:48:38.483 INFO:teuthology.orchestra.run.smithi139.stdout: "Firmware":"5.14.0-5", 2024-09-11T07:48:38.483 INFO:teuthology.orchestra.run.smithi139.stdout: "ModelNumber":"Linux", 2024-09-11T07:48:38.483 INFO:teuthology.orchestra.run.smithi139.stdout: "SerialNumber":"0100f88e8ad7605b83c5", 2024-09-11T07:48:38.483 INFO:teuthology.orchestra.run.smithi139.stdout: "UsedBytes":95995035648, 2024-09-11T07:48:38.483 INFO:teuthology.orchestra.run.smithi139.stdout: "MaximumLBA":187490304, 2024-09-11T07:48:38.483 INFO:teuthology.orchestra.run.smithi139.stdout: "PhysicalSize":95995035648, 2024-09-11T07:48:38.483 INFO:teuthology.orchestra.run.smithi139.stdout: "SectorSize":512 2024-09-11T07:48:38.483 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-11T07:48:38.483 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-11T07:48:38.483 INFO:teuthology.orchestra.run.smithi139.stdout: "NameSpace":1, 2024-09-11T07:48:38.483 INFO:teuthology.orchestra.run.smithi139.stdout: "DevicePath":"/dev/nvme2n1", 2024-09-11T07:48:38.483 INFO:teuthology.orchestra.run.smithi139.stdout: "GenericPath":"/dev/ng2n1", 2024-09-11T07:48:38.483 INFO:teuthology.orchestra.run.smithi139.stdout: "Firmware":"5.14.0-5", 2024-09-11T07:48:38.484 INFO:teuthology.orchestra.run.smithi139.stdout: "ModelNumber":"Linux", 2024-09-11T07:48:38.484 INFO:teuthology.orchestra.run.smithi139.stdout: "SerialNumber":"635643ae657864394059", 2024-09-11T07:48:38.484 INFO:teuthology.orchestra.run.smithi139.stdout: "UsedBytes":95995035648, 2024-09-11T07:48:38.484 INFO:teuthology.orchestra.run.smithi139.stdout: "MaximumLBA":187490304, 2024-09-11T07:48:38.484 INFO:teuthology.orchestra.run.smithi139.stdout: "PhysicalSize":95995035648, 2024-09-11T07:48:38.484 INFO:teuthology.orchestra.run.smithi139.stdout: "SectorSize":512 2024-09-11T07:48:38.484 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-11T07:48:38.484 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-11T07:48:38.484 INFO:teuthology.orchestra.run.smithi139.stdout: "NameSpace":1, 2024-09-11T07:48:38.484 INFO:teuthology.orchestra.run.smithi139.stdout: "DevicePath":"/dev/nvme1n1", 2024-09-11T07:48:38.484 INFO:teuthology.orchestra.run.smithi139.stdout: "GenericPath":"/dev/ng1n1", 2024-09-11T07:48:38.484 INFO:teuthology.orchestra.run.smithi139.stdout: "Firmware":"5.14.0-5", 2024-09-11T07:48:38.484 INFO:teuthology.orchestra.run.smithi139.stdout: "ModelNumber":"Linux", 2024-09-11T07:48:38.484 INFO:teuthology.orchestra.run.smithi139.stdout: "SerialNumber":"6c8cc02e8c293f0e3cb0", 2024-09-11T07:48:38.484 INFO:teuthology.orchestra.run.smithi139.stdout: "UsedBytes":95995035648, 2024-09-11T07:48:38.485 INFO:teuthology.orchestra.run.smithi139.stdout: "MaximumLBA":187490304, 2024-09-11T07:48:38.485 INFO:teuthology.orchestra.run.smithi139.stdout: "PhysicalSize":95995035648, 2024-09-11T07:48:38.485 INFO:teuthology.orchestra.run.smithi139.stdout: "SectorSize":512 2024-09-11T07:48:38.485 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-11T07:48:38.485 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-11T07:48:38.485 INFO:teuthology.orchestra.run.smithi139.stdout: "NameSpace":1, 2024-09-11T07:48:38.485 INFO:teuthology.orchestra.run.smithi139.stdout: "DevicePath":"/dev/nvme0n1", 2024-09-11T07:48:38.485 INFO:teuthology.orchestra.run.smithi139.stdout: "GenericPath":"/dev/ng0n1", 2024-09-11T07:48:38.485 INFO:teuthology.orchestra.run.smithi139.stdout: "Firmware":"8DV101H0", 2024-09-11T07:48:38.485 INFO:teuthology.orchestra.run.smithi139.stdout: "ModelNumber":"INTEL SSDPEDMD400G4", 2024-09-11T07:48:38.485 INFO:teuthology.orchestra.run.smithi139.stdout: "SerialNumber":"PHFT620500N6400BGN", 2024-09-11T07:48:38.485 INFO:teuthology.orchestra.run.smithi139.stdout: "UsedBytes":400088457216, 2024-09-11T07:48:38.485 INFO:teuthology.orchestra.run.smithi139.stdout: "MaximumLBA":781422768, 2024-09-11T07:48:38.486 INFO:teuthology.orchestra.run.smithi139.stdout: "PhysicalSize":400088457216, 2024-09-11T07:48:38.486 INFO:teuthology.orchestra.run.smithi139.stdout: "SectorSize":512 2024-09-11T07:48:38.486 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-11T07:48:38.486 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-11T07:48:38.486 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-11T07:48:38.487 INFO:tasks.nvme_loop:new_devs ['/dev/nvme4n1', '/dev/nvme3n1', '/dev/nvme2n1', '/dev/nvme1n1'] 2024-09-11T07:48:38.487 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-11T07:48:38.487 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/scratch_devs 2024-09-11T07:48:38.552 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-11T07:48:38.640 INFO:tasks.cephadm:Config: {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1, 'mgr/cephadm/use_agent': True}, 'global': {'mon election default strategy': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'MON_DOWN', 'mons down', 'mon down', 'out of quorum'], 'sha1': 'd0e6828a2016d48cf25ad84064e50742bb1c39b9'} 2024-09-11T07:48:38.640 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T07:48:38.640 INFO:tasks.cephadm:Cluster fsid is 41bd1096-7012-11ef-bcea-c7b262605968 2024-09-11T07:48:38.640 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-11T07:48:38.641 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.110', 'mon.c': '[v2:172.21.15.110:3301,v1:172.21.15.110:6790]', 'mon.b': '172.21.15.139'} 2024-09-11T07:48:38.641 INFO:tasks.cephadm:First mon is mon.a on smithi110 2024-09-11T07:48:38.641 INFO:tasks.cephadm:First mgr is y 2024-09-11T07:48:38.641 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-11T07:48:38.641 DEBUG:teuthology.orchestra.run.smithi110:> sudo hostname $(hostname -s) 2024-09-11T07:48:38.672 DEBUG:teuthology.orchestra.run.smithi139:> sudo hostname $(hostname -s) 2024-09-11T07:48:38.697 INFO:tasks.cephadm:Downloading cephadm (repo https://git.ceph.com/ceph-ci.git ref d0e6828a2016d48cf25ad84064e50742bb1c39b9)... 2024-09-11T07:48:38.697 DEBUG:teuthology.orchestra.run.smithi110:> git clone https://git.ceph.com/ceph-ci.git testrepo && cd testrepo && git show d0e6828a2016d48cf25ad84064e50742bb1c39b9:src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-11T07:48:38.729 INFO:teuthology.orchestra.run.smithi110.stderr:Cloning into 'testrepo'... 2024-09-11T07:49:36.873 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r--. 1 ubuntu ubuntu 378585 Sep 11 07:49 /home/ubuntu/cephtest/cephadm 2024-09-11T07:49:36.874 DEBUG:teuthology.orchestra.run.smithi139:> git clone https://git.ceph.com/ceph-ci.git testrepo && cd testrepo && git show d0e6828a2016d48cf25ad84064e50742bb1c39b9:src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-11T07:49:36.894 INFO:teuthology.orchestra.run.smithi139.stderr:Cloning into 'testrepo'... 2024-09-11T07:50:34.865 INFO:teuthology.orchestra.run.smithi139.stdout:-rw-r--r--. 1 ubuntu ubuntu 378585 Sep 11 07:50 /home/ubuntu/cephtest/cephadm 2024-09-11T07:50:34.866 DEBUG:teuthology.orchestra.run.smithi110:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-11T07:50:34.887 DEBUG:teuthology.orchestra.run.smithi139:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-11T07:50:34.936 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 on all hosts... 2024-09-11T07:50:34.936 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 pull 2024-09-11T07:50:34.938 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 pull 2024-09-11T07:50:36.659 INFO:teuthology.orchestra.run.smithi110.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9... 2024-09-11T07:50:38.349 INFO:teuthology.orchestra.run.smithi139.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9... 2024-09-11T07:51:21.260 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-11T07:51:21.260 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_version": "ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable)", 2024-09-11T07:51:21.260 INFO:teuthology.orchestra.run.smithi139.stdout: "image_id": "4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b", 2024-09-11T07:51:21.260 INFO:teuthology.orchestra.run.smithi139.stdout: "repo_digests": [ 2024-09-11T07:51:21.261 INFO:teuthology.orchestra.run.smithi139.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40" 2024-09-11T07:51:21.261 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-11T07:51:21.261 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-11T07:51:30.146 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-11T07:51:30.146 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_version": "ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable)", 2024-09-11T07:51:30.146 INFO:teuthology.orchestra.run.smithi110.stdout: "image_id": "4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b", 2024-09-11T07:51:30.146 INFO:teuthology.orchestra.run.smithi110.stdout: "repo_digests": [ 2024-09-11T07:51:30.146 INFO:teuthology.orchestra.run.smithi110.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40" 2024-09-11T07:51:30.146 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-11T07:51:30.147 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-11T07:51:30.164 DEBUG:teuthology.orchestra.run.smithi110:> sudo mkdir -p /etc/ceph 2024-09-11T07:51:30.224 DEBUG:teuthology.orchestra.run.smithi139:> sudo mkdir -p /etc/ceph 2024-09-11T07:51:30.257 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod 777 /etc/ceph 2024-09-11T07:51:30.281 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 777 /etc/ceph 2024-09-11T07:51:30.324 INFO:tasks.cephadm:Writing seed config... 2024-09-11T07:51:30.326 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-11T07:51:30.326 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-11T07:51:30.326 INFO:tasks.cephadm: override: [mgr] mgr/cephadm/use_agent = True 2024-09-11T07:51:30.326 INFO:tasks.cephadm: override: [global] mon election default strategy = 1 2024-09-11T07:51:30.326 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-11T07:51:30.326 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-11T07:51:30.326 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-11T07:51:30.326 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-11T07:51:30.326 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-11T07:51:30.326 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-11T07:51:30.327 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-11T07:51:30.327 DEBUG:teuthology.orchestra.run.smithi110:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-11T07:51:30.342 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 41bd1096-7012-11ef-bcea-c7b262605968 mon election default strategy = 1 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 mgr/cephadm/use_agent = True [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true 2024-09-11T07:51:30.342 DEBUG:teuthology.orchestra.run.smithi110:mon.a> sudo journalctl -f -n 0 -u ceph-41bd1096-7012-11ef-bcea-c7b262605968@mon.a.service 2024-09-11T07:51:30.386 DEBUG:teuthology.orchestra.run.smithi110:mgr.y> sudo journalctl -f -n 0 -u ceph-41bd1096-7012-11ef-bcea-c7b262605968@mgr.y.service 2024-09-11T07:51:30.429 INFO:tasks.cephadm:Bootstrapping... 2024-09-11T07:51:30.430 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 -v bootstrap --fsid 41bd1096-7012-11ef-bcea-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.110 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-11T07:51:30.626 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:51:30.626 INFO:teuthology.orchestra.run.smithi110.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9', '-v', 'bootstrap', '--fsid', '41bd1096-7012-11ef-bcea-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.110', '--skip-admin-label'] 2024-09-11T07:51:30.650 INFO:teuthology.orchestra.run.smithi110.stdout:/bin/podman: stdout 5.2.2 2024-09-11T07:51:30.650 INFO:teuthology.orchestra.run.smithi110.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-11T07:51:30.651 INFO:teuthology.orchestra.run.smithi110.stdout:Verifying podman|docker is present... 2024-09-11T07:51:30.672 INFO:teuthology.orchestra.run.smithi110.stdout:/bin/podman: stdout 5.2.2 2024-09-11T07:51:30.672 INFO:teuthology.orchestra.run.smithi110.stdout:Verifying lvm2 is present... 2024-09-11T07:51:30.672 INFO:teuthology.orchestra.run.smithi110.stdout:Verifying time synchronization is in place... 2024-09-11T07:51:30.680 INFO:teuthology.orchestra.run.smithi110.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-11T07:51:30.680 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-11T07:51:30.687 INFO:teuthology.orchestra.run.smithi110.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-11T07:51:30.687 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stdout inactive 2024-09-11T07:51:30.693 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stdout enabled 2024-09-11T07:51:30.700 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stdout active 2024-09-11T07:51:30.701 INFO:teuthology.orchestra.run.smithi110.stdout:Unit chronyd.service is enabled and running 2024-09-11T07:51:30.701 INFO:teuthology.orchestra.run.smithi110.stdout:Repeating the final host check... 2024-09-11T07:51:30.722 INFO:teuthology.orchestra.run.smithi110.stdout:/bin/podman: stdout 5.2.2 2024-09-11T07:51:30.722 INFO:teuthology.orchestra.run.smithi110.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-11T07:51:30.722 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl is present 2024-09-11T07:51:30.723 INFO:teuthology.orchestra.run.smithi110.stdout:lvcreate is present 2024-09-11T07:51:30.729 INFO:teuthology.orchestra.run.smithi110.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-11T07:51:30.729 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-11T07:51:30.736 INFO:teuthology.orchestra.run.smithi110.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-11T07:51:30.736 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stdout inactive 2024-09-11T07:51:30.743 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stdout enabled 2024-09-11T07:51:30.750 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stdout active 2024-09-11T07:51:30.750 INFO:teuthology.orchestra.run.smithi110.stdout:Unit chronyd.service is enabled and running 2024-09-11T07:51:30.750 INFO:teuthology.orchestra.run.smithi110.stdout:Host looks OK 2024-09-11T07:51:30.750 INFO:teuthology.orchestra.run.smithi110.stdout:Cluster fsid: 41bd1096-7012-11ef-bcea-c7b262605968 2024-09-11T07:51:30.750 INFO:teuthology.orchestra.run.smithi110.stdout:Acquiring lock 139741652771216 on /run/cephadm/41bd1096-7012-11ef-bcea-c7b262605968.lock 2024-09-11T07:51:30.750 INFO:teuthology.orchestra.run.smithi110.stdout:Lock 139741652771216 acquired on /run/cephadm/41bd1096-7012-11ef-bcea-c7b262605968.lock 2024-09-11T07:51:30.750 INFO:teuthology.orchestra.run.smithi110.stdout:Verifying IP 172.21.15.110 port 3300 ... 2024-09-11T07:51:30.751 INFO:teuthology.orchestra.run.smithi110.stdout:Verifying IP 172.21.15.110 port 6789 ... 2024-09-11T07:51:30.751 INFO:teuthology.orchestra.run.smithi110.stdout:Base mon IP(s) is [172.21.15.110:3300, 172.21.15.110:6789], mon addrv is [v2:172.21.15.110:3300,v1:172.21.15.110:6789] 2024-09-11T07:51:30.754 INFO:teuthology.orchestra.run.smithi110.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.110 metric 100 2024-09-11T07:51:30.754 INFO:teuthology.orchestra.run.smithi110.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.110 metric 100 2024-09-11T07:51:30.757 INFO:teuthology.orchestra.run.smithi110.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-11T07:51:30.757 INFO:teuthology.orchestra.run.smithi110.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-11T07:51:30.757 INFO:teuthology.orchestra.run.smithi110.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-11T07:51:30.759 INFO:teuthology.orchestra.run.smithi110.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-11T07:51:30.759 INFO:teuthology.orchestra.run.smithi110.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-11T07:51:30.760 INFO:teuthology.orchestra.run.smithi110.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-11T07:51:30.760 INFO:teuthology.orchestra.run.smithi110.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-11T07:51:30.760 INFO:teuthology.orchestra.run.smithi110.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:cc35/64 scope link noprefixroute 2024-09-11T07:51:30.760 INFO:teuthology.orchestra.run.smithi110.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-11T07:51:30.761 INFO:teuthology.orchestra.run.smithi110.stdout:Mon IP `172.21.15.110` is in CIDR network `172.21.0.0/20` 2024-09-11T07:51:30.761 INFO:teuthology.orchestra.run.smithi110.stdout:Mon IP `172.21.15.110` is in CIDR network `172.21.0.0/20` 2024-09-11T07:51:30.761 INFO:teuthology.orchestra.run.smithi110.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-11T07:51:30.761 INFO:teuthology.orchestra.run.smithi110.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-11T07:51:30.761 INFO:teuthology.orchestra.run.smithi110.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9... 2024-09-11T07:51:31.221 INFO:teuthology.orchestra.run.smithi110.stdout:/bin/podman: stdout 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b 2024-09-11T07:51:31.221 INFO:teuthology.orchestra.run.smithi110.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9... 2024-09-11T07:51:31.221 INFO:teuthology.orchestra.run.smithi110.stdout:/bin/podman: stderr Getting image source signatures 2024-09-11T07:51:31.221 INFO:teuthology.orchestra.run.smithi110.stdout:/bin/podman: stderr Copying blob sha256:0d2f264d63cec2f6fb0f85eed7b2c8c5b4db3ab9edd9d71a74c04647294a553f 2024-09-11T07:51:31.221 INFO:teuthology.orchestra.run.smithi110.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-11T07:51:31.222 INFO:teuthology.orchestra.run.smithi110.stdout:/bin/podman: stderr Copying config sha256:4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b 2024-09-11T07:51:31.222 INFO:teuthology.orchestra.run.smithi110.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-11T07:51:32.151 INFO:teuthology.orchestra.run.smithi110.stdout:ceph: stdout ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable) 2024-09-11T07:51:32.151 INFO:teuthology.orchestra.run.smithi110.stdout:Ceph version: ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable) 2024-09-11T07:51:32.151 INFO:teuthology.orchestra.run.smithi110.stdout:Extracting ceph user uid/gid from container image... 2024-09-11T07:51:33.125 INFO:teuthology.orchestra.run.smithi110.stdout:stat: stdout 167 167 2024-09-11T07:51:33.125 INFO:teuthology.orchestra.run.smithi110.stdout:Creating initial keys... 2024-09-11T07:51:34.264 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph-authtool: stdout AQAFTOFmvoYbGhAA1oV6GKP9AacCx8eNlbWl6w== 2024-09-11T07:51:35.311 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph-authtool: stdout AQAGTOFmFwvkIxAAfCOgYS53dURrvxKQ1O28aw== 2024-09-11T07:51:36.661 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph-authtool: stdout AQAHTOFmWsQ+JRAA4kZUva386zeG7/7pZt3cyg== 2024-09-11T07:51:36.661 INFO:teuthology.orchestra.run.smithi110.stdout:Creating initial monmap... 2024-09-11T07:51:46.409 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-11T07:51:46.410 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = octopus 2024-09-11T07:51:46.410 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 41bd1096-7012-11ef-bcea-c7b262605968 2024-09-11T07:51:46.410 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-11T07:51:46.410 INFO:teuthology.orchestra.run.smithi110.stdout:monmaptool for a [v2:172.21.15.110:3300,v1:172.21.15.110:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-11T07:51:46.410 INFO:teuthology.orchestra.run.smithi110.stdout:setting min_mon_release = octopus 2024-09-11T07:51:46.410 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/monmaptool: set fsid to 41bd1096-7012-11ef-bcea-c7b262605968 2024-09-11T07:51:46.410 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-11T07:51:46.410 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T07:51:46.410 INFO:teuthology.orchestra.run.smithi110.stdout:Creating mon... 2024-09-11T07:51:47.399 INFO:teuthology.orchestra.run.smithi110.stdout:create mon.a on 2024-09-11T07:51:47.758 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-11T07:51:47.915 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-41bd1096-7012-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-41bd1096-7012-11ef-bcea-c7b262605968.target. 2024-09-11T07:51:47.915 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-41bd1096-7012-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-41bd1096-7012-11ef-bcea-c7b262605968.target. 2024-09-11T07:51:48.100 INFO:teuthology.orchestra.run.smithi110.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-41bd1096-7012-11ef-bcea-c7b262605968@mon.a 2024-09-11T07:51:48.100 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stderr Failed to reset failed state of unit ceph-41bd1096-7012-11ef-bcea-c7b262605968@mon.a.service: Unit ceph-41bd1096-7012-11ef-bcea-c7b262605968@mon.a.service not loaded. 2024-09-11T07:51:48.274 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-41bd1096-7012-11ef-bcea-c7b262605968.target.wants/ceph-41bd1096-7012-11ef-bcea-c7b262605968@mon.a.service → /etc/systemd/system/ceph-41bd1096-7012-11ef-bcea-c7b262605968@.service. 2024-09-11T07:51:48.849 INFO:teuthology.orchestra.run.smithi110.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T07:51:48.849 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stdout disabled 2024-09-11T07:51:48.857 INFO:teuthology.orchestra.run.smithi110.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T07:51:48.857 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stdout inactive 2024-09-11T07:51:48.857 INFO:teuthology.orchestra.run.smithi110.stdout:firewalld.service is not enabled 2024-09-11T07:51:48.857 INFO:teuthology.orchestra.run.smithi110.stdout:Not possible to enable service . firewalld.service is not available 2024-09-11T07:51:48.857 INFO:teuthology.orchestra.run.smithi110.stdout:Waiting for mon to start... 2024-09-11T07:51:48.858 INFO:teuthology.orchestra.run.smithi110.stdout:Waiting for mon... 2024-09-11T07:51:48.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:48 smithi110 systemd[1]: Started Ceph mon.a for 41bd1096-7012-11ef-bcea-c7b262605968. 2024-09-11T07:51:49.879 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout cluster: 2024-09-11T07:51:49.879 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout id: 41bd1096-7012-11ef-bcea-c7b262605968 2024-09-11T07:51:49.879 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-11T07:51:49.879 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout 2024-09-11T07:51:49.879 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout services: 2024-09-11T07:51:49.879 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.615413s) 2024-09-11T07:51:49.879 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-11T07:51:49.879 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-11T07:51:49.879 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout 2024-09-11T07:51:49.879 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout data: 2024-09-11T07:51:49.879 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-11T07:51:49.880 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-11T07:51:49.880 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-11T07:51:49.880 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout pgs: 2024-09-11T07:51:49.880 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout 2024-09-11T07:51:49.880 INFO:teuthology.orchestra.run.smithi110.stdout:mon is available 2024-09-11T07:51:49.880 INFO:teuthology.orchestra.run.smithi110.stdout:Assimilating anything we can from ceph.conf... 2024-09-11T07:51:49.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:49 smithi110 ceph-mon[25029]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-11T07:51:49.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:49 smithi110 ceph-mon[25029]: monmap e1: 1 mons at {a=[v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-11T07:51:49.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:49 smithi110 ceph-mon[25029]: fsmap 2024-09-11T07:51:49.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:49 smithi110 ceph-mon[25029]: osdmap e1: 0 total, 0 up, 0 in 2024-09-11T07:51:49.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:49 smithi110 ceph-mon[25029]: mgrmap e1: no daemons active 2024-09-11T07:51:49.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:49 smithi110 ceph-mon[25029]: from='client.? 172.21.15.110:0/88472234' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-11T07:51:50.885 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout 2024-09-11T07:51:50.885 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout [global] 2024-09-11T07:51:50.885 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout fsid = 41bd1096-7012-11ef-bcea-c7b262605968 2024-09-11T07:51:50.885 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-11T07:51:50.885 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.110:3300,v1:172.21.15.110:6789] 2024-09-11T07:51:50.885 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-11T07:51:50.885 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-11T07:51:50.886 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-11T07:51:50.886 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-11T07:51:50.886 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout 2024-09-11T07:51:50.886 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-11T07:51:50.886 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout mgr/cephadm/use_agent = True 2024-09-11T07:51:50.886 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-11T07:51:50.886 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout 2024-09-11T07:51:50.886 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout [osd] 2024-09-11T07:51:50.886 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-11T07:51:50.886 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-11T07:51:50.886 INFO:teuthology.orchestra.run.smithi110.stdout:Generating new minimal ceph.conf... 2024-09-11T07:51:50.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:50 smithi110 ceph-mon[25029]: from='client.? 172.21.15.110:0/3879216670' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-11T07:51:50.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:50 smithi110 ceph-mon[25029]: from='client.? 172.21.15.110:0/3879216670' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-11T07:51:52.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:51 smithi110 ceph-mon[25029]: from='client.? 172.21.15.110:0/2454654368' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:51:52.706 INFO:teuthology.orchestra.run.smithi110.stdout:Restarting the monitor... 2024-09-11T07:51:52.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:52 smithi110 systemd[1]: Stopping Ceph mon.a for 41bd1096-7012-11ef-bcea-c7b262605968... 2024-09-11T07:51:53.357 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:52 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mon-a[25023]: 2024-09-11T07:51:52.988+0000 7f1703c07640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T07:51:53.357 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:52 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mon-a[25023]: 2024-09-11T07:51:52.988+0000 7f1703c07640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-11T07:51:53.357 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:53 smithi110 podman[25270]: 2024-09-11 07:51:53.090790943 +0000 UTC m=+0.218054481 container died 98ba207a0f484deb255aef87be5d25272831fa288c70f1cdbd44f889a2a422c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-mon-a, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T07:51:53.710 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:53 smithi110 podman[25270]: 2024-09-11 07:51:53.446697917 +0000 UTC m=+0.573961454 container cleanup 98ba207a0f484deb255aef87be5d25272831fa288c70f1cdbd44f889a2a422c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-11T07:51:53.710 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:53 smithi110 bash[25270]: ceph-41bd1096-7012-11ef-bcea-c7b262605968-mon-a 2024-09-11T07:51:53.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:53 smithi110 podman[25283]: 2024-09-11 07:51:53.70920183 +0000 UTC m=+0.621012158 container remove 98ba207a0f484deb255aef87be5d25272831fa288c70f1cdbd44f889a2a422c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-mon-a, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True) 2024-09-11T07:51:53.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:53 smithi110 systemd[1]: ceph-41bd1096-7012-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-11T07:51:53.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:53 smithi110 systemd[1]: Stopped Ceph mon.a for 41bd1096-7012-11ef-bcea-c7b262605968. 2024-09-11T07:51:53.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:53 smithi110 systemd[1]: Starting Ceph mon.a for 41bd1096-7012-11ef-bcea-c7b262605968... 2024-09-11T07:51:54.274 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 podman[25372]: 2024-09-11 07:51:54.007001202 +0000 UTC m=+0.089865600 container create eb9b8d53622fef5a3fddafcb96b4053cf087eee1fa8873ff87dbf3ecbc2aafae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T07:51:54.275 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 podman[25372]: 2024-09-11 07:51:53.932734304 +0000 UTC m=+0.015598703 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T07:51:54.275 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 podman[25372]: 2024-09-11 07:51:54.240351087 +0000 UTC m=+0.323215489 container init eb9b8d53622fef5a3fddafcb96b4053cf087eee1fa8873ff87dbf3ecbc2aafae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-mon-a, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-11T07:51:54.275 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 podman[25372]: 2024-09-11 07:51:54.243564468 +0000 UTC m=+0.326428870 container start eb9b8d53622fef5a3fddafcb96b4053cf087eee1fa8873ff87dbf3ecbc2aafae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True) 2024-09-11T07:51:54.275 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: set uid:gid to 167:167 (ceph:ceph) 2024-09-11T07:51:54.275 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable), process ceph-mon, pid 2 2024-09-11T07:51:54.275 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: pidfile_write: ignore empty --pid-file 2024-09-11T07:51:54.489 INFO:teuthology.orchestra.run.smithi110.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-11T07:51:54.612 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: load: jerasure load: lrc 2024-09-11T07:51:54.612 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: RocksDB version: 6.15.5 2024-09-11T07:51:54.612 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-09-11T07:51:54.612 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Compile date Sep 10 2024 2024-09-11T07:51:54.612 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: DB SUMMARY 2024-09-11T07:51:54.612 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: DB Session ID: GJ0EQILQDBVJUD7RXIFD 2024-09-11T07:51:54.612 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: CURRENT file: CURRENT 2024-09-11T07:51:54.612 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: IDENTITY file: IDENTITY 2024-09-11T07:51:54.612 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: MANIFEST file: MANIFEST-000009 size: 131 Bytes 2024-09-11T07:51:54.612 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-11T07:51:54.613 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000010.log size: 82880 ; 2024-09-11T07:51:54.613 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.error_if_exists: 0 2024-09-11T07:51:54.613 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.create_if_missing: 0 2024-09-11T07:51:54.613 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.paranoid_checks: 1 2024-09-11T07:51:54.613 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-11T07:51:54.613 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.env: 0x55c5c9a1a440 2024-09-11T07:51:54.613 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.fs: Posix File System 2024-09-11T07:51:54.613 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.info_log: 0x55c5ca662ec0 2024-09-11T07:51:54.613 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.max_file_opening_threads: 16 2024-09-11T07:51:54.613 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.statistics: (nil) 2024-09-11T07:51:54.613 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.use_fsync: 0 2024-09-11T07:51:54.613 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.max_log_file_size: 0 2024-09-11T07:51:54.613 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-11T07:51:54.613 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-11T07:51:54.614 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.keep_log_file_num: 1000 2024-09-11T07:51:54.614 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.recycle_log_file_num: 0 2024-09-11T07:51:54.614 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.allow_fallocate: 1 2024-09-11T07:51:54.614 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.allow_mmap_reads: 0 2024-09-11T07:51:54.614 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.allow_mmap_writes: 0 2024-09-11T07:51:54.614 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.use_direct_reads: 0 2024-09-11T07:51:54.614 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-11T07:51:54.614 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.create_missing_column_families: 0 2024-09-11T07:51:54.614 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.db_log_dir: 2024-09-11T07:51:54.614 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-a/store.db 2024-09-11T07:51:54.614 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-11T07:51:54.614 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-11T07:51:54.614 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-11T07:51:54.614 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-11T07:51:54.615 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-11T07:51:54.616 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-11T07:51:54.616 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.advise_random_on_open: 1 2024-09-11T07:51:54.616 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.db_write_buffer_size: 0 2024-09-11T07:51:54.616 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.write_buffer_manager: 0x55c5ca720780 2024-09-11T07:51:54.616 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-11T07:51:54.616 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-09-11T07:51:54.616 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-11T07:51:54.616 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-11T07:51:54.617 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.rate_limiter: (nil) 2024-09-11T07:51:54.617 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-11T07:51:54.617 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.wal_recovery_mode: 2 2024-09-11T07:51:54.617 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.enable_thread_tracking: 0 2024-09-11T07:51:54.617 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.enable_pipelined_write: 0 2024-09-11T07:51:54.617 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.unordered_write: 0 2024-09-11T07:51:54.617 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-11T07:51:54.617 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-11T07:51:54.617 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-11T07:51:54.617 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-11T07:51:54.617 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.row_cache: None 2024-09-11T07:51:54.617 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.wal_filter: None 2024-09-11T07:51:54.617 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-11T07:51:54.617 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.allow_ingest_behind: 0 2024-09-11T07:51:54.618 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.preserve_deletes: 0 2024-09-11T07:51:54.618 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.two_write_queues: 0 2024-09-11T07:51:54.618 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.manual_wal_flush: 0 2024-09-11T07:51:54.618 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.atomic_flush: 0 2024-09-11T07:51:54.618 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-11T07:51:54.618 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-11T07:51:54.618 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-11T07:51:54.618 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.log_readahead_size: 0 2024-09-11T07:51:54.618 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-11T07:51:54.618 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.best_efforts_recovery: 0 2024-09-11T07:51:54.618 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-11T07:51:54.618 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-11T07:51:54.618 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.allow_data_in_errors: 0 2024-09-11T07:51:54.618 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.db_host_id: __hostname__ 2024-09-11T07:51:54.619 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.max_background_jobs: 2 2024-09-11T07:51:54.619 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.max_background_compactions: -1 2024-09-11T07:51:54.619 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.max_subcompactions: 1 2024-09-11T07:51:54.619 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-11T07:51:54.619 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-11T07:51:54.619 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-11T07:51:54.619 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.max_total_wal_size: 0 2024-09-11T07:51:54.619 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-11T07:51:54.619 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-11T07:51:54.619 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-11T07:51:54.619 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-11T07:51:54.619 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.max_open_files: -1 2024-09-11T07:51:54.619 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.bytes_per_sync: 0 2024-09-11T07:51:54.619 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-11T07:51:54.620 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-11T07:51:54.620 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.compaction_readahead_size: 0 2024-09-11T07:51:54.620 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.max_background_flushes: -1 2024-09-11T07:51:54.620 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Compression algorithms supported: 2024-09-11T07:51:54.620 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: kZSTD supported: 0 2024-09-11T07:51:54.620 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: kXpressCompression supported: 0 2024-09-11T07:51:54.620 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: kBZip2Compression supported: 0 2024-09-11T07:51:54.620 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-11T07:51:54.620 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: kLZ4Compression supported: 1 2024-09-11T07:51:54.620 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: kZlibCompression supported: 1 2024-09-11T07:51:54.620 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: kLZ4HCCompression supported: 1 2024-09-11T07:51:54.620 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: kSnappyCompression supported: 1 2024-09-11T07:51:54.620 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-11T07:51:54.620 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000009 2024-09-11T07:51:54.621 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-09-11T07:51:54.621 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-11T07:51:54.621 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.merge_operator: 2024-09-11T07:51:54.621 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.compaction_filter: None 2024-09-11T07:51:54.621 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.compaction_filter_factory: None 2024-09-11T07:51:54.621 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.sst_partitioner_factory: None 2024-09-11T07:51:54.621 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-11T07:51:54.621 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-11T07:51:54.621 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55c5ca609a38) 2024-09-11T07:51:54.621 INFO:journalctl@ceph.mon.a.smithi110.stdout: cache_index_and_filter_blocks: 1 2024-09-11T07:51:54.621 INFO:journalctl@ceph.mon.a.smithi110.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-11T07:51:54.621 INFO:journalctl@ceph.mon.a.smithi110.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-11T07:51:54.621 INFO:journalctl@ceph.mon.a.smithi110.stdout: pin_top_level_index_and_filter: 1 2024-09-11T07:51:54.621 INFO:journalctl@ceph.mon.a.smithi110.stdout: index_type: 0 2024-09-11T07:51:54.621 INFO:journalctl@ceph.mon.a.smithi110.stdout: data_block_index_type: 0 2024-09-11T07:51:54.622 INFO:journalctl@ceph.mon.a.smithi110.stdout: index_shortening: 1 2024-09-11T07:51:54.622 INFO:journalctl@ceph.mon.a.smithi110.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-11T07:51:54.622 INFO:journalctl@ceph.mon.a.smithi110.stdout: hash_index_allow_collision: 1 2024-09-11T07:51:54.622 INFO:journalctl@ceph.mon.a.smithi110.stdout: checksum: 1 2024-09-11T07:51:54.622 INFO:journalctl@ceph.mon.a.smithi110.stdout: no_block_cache: 0 2024-09-11T07:51:54.622 INFO:journalctl@ceph.mon.a.smithi110.stdout: block_cache: 0x55c5ca63f350 2024-09-11T07:51:54.622 INFO:journalctl@ceph.mon.a.smithi110.stdout: block_cache_name: BinnedLRUCache 2024-09-11T07:51:54.622 INFO:journalctl@ceph.mon.a.smithi110.stdout: block_cache_options: 2024-09-11T07:51:54.622 INFO:journalctl@ceph.mon.a.smithi110.stdout: capacity : 536870912 2024-09-11T07:51:54.622 INFO:journalctl@ceph.mon.a.smithi110.stdout: num_shard_bits : 4 2024-09-11T07:51:54.622 INFO:journalctl@ceph.mon.a.smithi110.stdout: strict_capacity_limit : 0 2024-09-11T07:51:54.622 INFO:journalctl@ceph.mon.a.smithi110.stdout: high_pri_pool_ratio: 0.000 2024-09-11T07:51:54.622 INFO:journalctl@ceph.mon.a.smithi110.stdout: block_cache_compressed: (nil) 2024-09-11T07:51:54.623 INFO:journalctl@ceph.mon.a.smithi110.stdout: persistent_cache: (nil) 2024-09-11T07:51:54.623 INFO:journalctl@ceph.mon.a.smithi110.stdout: block_size: 4096 2024-09-11T07:51:54.623 INFO:journalctl@ceph.mon.a.smithi110.stdout: block_size_deviation: 10 2024-09-11T07:51:54.623 INFO:journalctl@ceph.mon.a.smithi110.stdout: block_restart_interval: 16 2024-09-11T07:51:54.623 INFO:journalctl@ceph.mon.a.smithi110.stdout: index_block_restart_interval: 1 2024-09-11T07:51:54.623 INFO:journalctl@ceph.mon.a.smithi110.stdout: metadata_block_size: 4096 2024-09-11T07:51:54.623 INFO:journalctl@ceph.mon.a.smithi110.stdout: partition_filters: 0 2024-09-11T07:51:54.623 INFO:journalctl@ceph.mon.a.smithi110.stdout: use_delta_encoding: 1 2024-09-11T07:51:54.623 INFO:journalctl@ceph.mon.a.smithi110.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-09-11T07:51:54.623 INFO:journalctl@ceph.mon.a.smithi110.stdout: whole_key_filtering: 1 2024-09-11T07:51:54.623 INFO:journalctl@ceph.mon.a.smithi110.stdout: verify_compression: 0 2024-09-11T07:51:54.623 INFO:journalctl@ceph.mon.a.smithi110.stdout: read_amp_bytes_per_bit: 0 2024-09-11T07:51:54.623 INFO:journalctl@ceph.mon.a.smithi110.stdout: format_version: 4 2024-09-11T07:51:54.624 INFO:journalctl@ceph.mon.a.smithi110.stdout: enable_index_compression: 1 2024-09-11T07:51:54.624 INFO:journalctl@ceph.mon.a.smithi110.stdout: block_align: 0 2024-09-11T07:51:54.624 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.write_buffer_size: 33554432 2024-09-11T07:51:54.624 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.max_write_buffer_number: 2 2024-09-11T07:51:54.624 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.compression: NoCompression 2024-09-11T07:51:54.624 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.bottommost_compression: Disabled 2024-09-11T07:51:54.624 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.prefix_extractor: nullptr 2024-09-11T07:51:54.624 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-11T07:51:54.624 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.num_levels: 7 2024-09-11T07:51:54.624 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-11T07:51:54.624 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-11T07:51:54.624 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-11T07:51:54.624 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-11T07:51:54.624 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-11T07:51:54.625 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-11T07:51:54.625 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-11T07:51:54.625 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-11T07:51:54.625 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-11T07:51:54.625 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-11T07:51:54.625 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-11T07:51:54.625 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.compression_opts.level: 32767 2024-09-11T07:51:54.625 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.compression_opts.strategy: 0 2024-09-11T07:51:54.625 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-11T07:51:54.625 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-11T07:51:54.625 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-11T07:51:54.625 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.compression_opts.enabled: false 2024-09-11T07:51:54.625 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-11T07:51:54.625 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-11T07:51:54.626 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-11T07:51:54.626 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.target_file_size_base: 67108864 2024-09-11T07:51:54.626 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-11T07:51:54.626 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-11T07:51:54.626 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-11T07:51:54.626 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-11T07:51:54.626 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-11T07:51:54.626 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-11T07:51:54.626 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-11T07:51:54.626 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-11T07:51:54.626 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-11T07:51:54.626 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-11T07:51:54.626 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-11T07:51:54.626 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-11T07:51:54.627 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-11T07:51:54.627 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.arena_block_size: 4194304 2024-09-11T07:51:54.627 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-11T07:51:54.627 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-11T07:51:54.627 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-09-11T07:51:54.627 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.disable_auto_compactions: 0 2024-09-11T07:51:54.627 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-11T07:51:54.627 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-11T07:51:54.627 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-11T07:51:54.627 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-11T07:51:54.627 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-11T07:51:54.627 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-11T07:51:54.627 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-11T07:51:54.627 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-11T07:51:54.628 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-11T07:51:54.628 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-11T07:51:54.628 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-11T07:51:54.628 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.inplace_update_support: 0 2024-09-11T07:51:54.628 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-11T07:51:54.628 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-11T07:51:54.628 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-11T07:51:54.628 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-11T07:51:54.628 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.bloom_locality: 0 2024-09-11T07:51:54.628 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.max_successive_merges: 0 2024-09-11T07:51:54.628 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-11T07:51:54.628 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.paranoid_file_checks: 0 2024-09-11T07:51:54.628 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.force_consistency_checks: 1 2024-09-11T07:51:54.628 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.report_bg_io_stats: 0 2024-09-11T07:51:54.629 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.ttl: 2592000 2024-09-11T07:51:54.629 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-11T07:51:54.629 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.enable_blob_files: false 2024-09-11T07:51:54.629 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.min_blob_size: 0 2024-09-11T07:51:54.629 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.blob_file_size: 268435456 2024-09-11T07:51:54.629 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-11T07:51:54.629 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-11T07:51:54.629 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-11T07:51:54.629 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 11, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-11T07:51:54.629 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 5 2024-09-11T07:51:54.629 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: [db/version_set.cc:4082] Creating manifest 13 2024-09-11T07:51:54.629 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726041114279162, "job": 1, "event": "recovery_started", "wal_files": [10]} 2024-09-11T07:51:54.629 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #10 mode 2 2024-09-11T07:51:54.629 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-09-11T07:51:54.630 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726041114279905, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 14, "file_size": 79610, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 77842, "index_size": 230, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 581, "raw_key_size": 9759, "raw_average_key_size": 47, "raw_value_size": 72249, "raw_average_value_size": 349, "num_data_blocks": 10, "num_entries": 207, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1726041114, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "b02ad482-dc59-4900-9d78-799cc2e5cb60", "db_session_id": "GJ0EQILQDBVJUD7RXIFD"}} 2024-09-11T07:51:54.630 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: [db/version_set.cc:4082] Creating manifest 15 2024-09-11T07:51:54.630 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726041114280255, "job": 1, "event": "recovery_finished"} 2024-09-11T07:51:54.630 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000010.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-11T07:51:54.630 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x55c5ca672700 2024-09-11T07:51:54.630 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: DB pointer 0x55c5ca642000 2024-09-11T07:51:54.630 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-09-11T07:51:54.630 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-09-11T07:51:54.630 INFO:journalctl@ceph.mon.a.smithi110.stdout: ** DB Stats ** 2024-09-11T07:51:54.630 INFO:journalctl@ceph.mon.a.smithi110.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T07:51:54.630 INFO:journalctl@ceph.mon.a.smithi110.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-11T07:51:54.630 INFO:journalctl@ceph.mon.a.smithi110.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-11T07:51:54.630 INFO:journalctl@ceph.mon.a.smithi110.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-11T07:51:54.630 INFO:journalctl@ceph.mon.a.smithi110.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-11T07:51:54.631 INFO:journalctl@ceph.mon.a.smithi110.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-09-11T07:51:54.631 INFO:journalctl@ceph.mon.a.smithi110.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-11T07:51:54.631 INFO:journalctl@ceph.mon.a.smithi110.stdout: 2024-09-11T07:51:54.631 INFO:journalctl@ceph.mon.a.smithi110.stdout: ** Compaction Stats [default] ** 2024-09-11T07:51:54.631 INFO:journalctl@ceph.mon.a.smithi110.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T07:51:54.631 INFO:journalctl@ceph.mon.a.smithi110.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T07:51:54.631 INFO:journalctl@ceph.mon.a.smithi110.stdout: L0 2/0 79.53 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 148.9 0.00 0.00 1 0.001 0 0 2024-09-11T07:51:54.631 INFO:journalctl@ceph.mon.a.smithi110.stdout: Sum 2/0 79.53 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 148.9 0.00 0.00 1 0.001 0 0 2024-09-11T07:51:54.631 INFO:journalctl@ceph.mon.a.smithi110.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 148.9 0.00 0.00 1 0.001 0 0 2024-09-11T07:51:54.631 INFO:journalctl@ceph.mon.a.smithi110.stdout: 2024-09-11T07:51:54.631 INFO:journalctl@ceph.mon.a.smithi110.stdout: ** Compaction Stats [default] ** 2024-09-11T07:51:54.631 INFO:journalctl@ceph.mon.a.smithi110.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T07:51:54.631 INFO:journalctl@ceph.mon.a.smithi110.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T07:51:54.631 INFO:journalctl@ceph.mon.a.smithi110.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 148.9 0.00 0.00 1 0.001 0 0 2024-09-11T07:51:54.631 INFO:journalctl@ceph.mon.a.smithi110.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T07:51:54.632 INFO:journalctl@ceph.mon.a.smithi110.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-11T07:51:54.632 INFO:journalctl@ceph.mon.a.smithi110.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-11T07:51:54.632 INFO:journalctl@ceph.mon.a.smithi110.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-11T07:51:54.632 INFO:journalctl@ceph.mon.a.smithi110.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-11T07:51:54.632 INFO:journalctl@ceph.mon.a.smithi110.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-11T07:51:54.632 INFO:journalctl@ceph.mon.a.smithi110.stdout: Cumulative compaction: 0.00 GB write, 24.70 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T07:51:54.632 INFO:journalctl@ceph.mon.a.smithi110.stdout: Interval compaction: 0.00 GB write, 24.70 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T07:51:54.632 INFO:journalctl@ceph.mon.a.smithi110.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-11T07:51:54.632 INFO:journalctl@ceph.mon.a.smithi110.stdout: 2024-09-11T07:51:54.632 INFO:journalctl@ceph.mon.a.smithi110.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-11T07:51:54.632 INFO:journalctl@ceph.mon.a.smithi110.stdout: 2024-09-11T07:51:54.632 INFO:journalctl@ceph.mon.a.smithi110.stdout: ** Compaction Stats [default] ** 2024-09-11T07:51:54.632 INFO:journalctl@ceph.mon.a.smithi110.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T07:51:54.633 INFO:journalctl@ceph.mon.a.smithi110.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T07:51:54.633 INFO:journalctl@ceph.mon.a.smithi110.stdout: L0 2/0 79.53 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 148.9 0.00 0.00 1 0.001 0 0 2024-09-11T07:51:54.633 INFO:journalctl@ceph.mon.a.smithi110.stdout: Sum 2/0 79.53 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 148.9 0.00 0.00 1 0.001 0 0 2024-09-11T07:51:54.633 INFO:journalctl@ceph.mon.a.smithi110.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-09-11T07:51:54.633 INFO:journalctl@ceph.mon.a.smithi110.stdout: 2024-09-11T07:51:54.633 INFO:journalctl@ceph.mon.a.smithi110.stdout: ** Compaction Stats [default] ** 2024-09-11T07:51:54.633 INFO:journalctl@ceph.mon.a.smithi110.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T07:51:54.633 INFO:journalctl@ceph.mon.a.smithi110.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T07:51:54.633 INFO:journalctl@ceph.mon.a.smithi110.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 148.9 0.00 0.00 1 0.001 0 0 2024-09-11T07:51:54.633 INFO:journalctl@ceph.mon.a.smithi110.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T07:51:54.634 INFO:journalctl@ceph.mon.a.smithi110.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-11T07:51:54.634 INFO:journalctl@ceph.mon.a.smithi110.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-11T07:51:54.634 INFO:journalctl@ceph.mon.a.smithi110.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-11T07:51:54.634 INFO:journalctl@ceph.mon.a.smithi110.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-11T07:51:54.634 INFO:journalctl@ceph.mon.a.smithi110.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-11T07:51:54.634 INFO:journalctl@ceph.mon.a.smithi110.stdout: Cumulative compaction: 0.00 GB write, 24.45 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T07:51:54.634 INFO:journalctl@ceph.mon.a.smithi110.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T07:51:54.634 INFO:journalctl@ceph.mon.a.smithi110.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-11T07:51:54.634 INFO:journalctl@ceph.mon.a.smithi110.stdout: 2024-09-11T07:51:54.634 INFO:journalctl@ceph.mon.a.smithi110.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-11T07:51:54.634 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: starting mon.a rank 0 at public addrs [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] at bind addrs [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 41bd1096-7012-11ef-bcea-c7b262605968 2024-09-11T07:51:54.634 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: mon.a@-1(???) e1 preinit fsid 41bd1096-7012-11ef-bcea-c7b262605968 2024-09-11T07:51:54.634 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: mon.a@-1(???).mds e1 new map 2024-09-11T07:51:54.634 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: mon.a@-1(???).mds e1 print_map 2024-09-11T07:51:54.635 INFO:journalctl@ceph.mon.a.smithi110.stdout: e1 2024-09-11T07:51:54.635 INFO:journalctl@ceph.mon.a.smithi110.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-11T07:51:54.635 INFO:journalctl@ceph.mon.a.smithi110.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-11T07:51:54.635 INFO:journalctl@ceph.mon.a.smithi110.stdout: legacy client fscid: -1 2024-09-11T07:51:54.635 INFO:journalctl@ceph.mon.a.smithi110.stdout: 2024-09-11T07:51:54.635 INFO:journalctl@ceph.mon.a.smithi110.stdout: No filesystems configured 2024-09-11T07:51:54.635 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-11T07:51:54.635 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T07:51:54.635 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T07:51:54.635 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T07:51:54.635 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-11T07:51:54.635 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-11T07:51:54.635 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: monmap e1: 1 mons at {a=[v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-11T07:51:54.636 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: fsmap 2024-09-11T07:51:54.636 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: osdmap e1: 0 total, 0 up, 0 in 2024-09-11T07:51:54.636 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 ceph-mon[25392]: mgrmap e1: no daemons active 2024-09-11T07:51:54.636 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 bash[25372]: eb9b8d53622fef5a3fddafcb96b4053cf087eee1fa8873ff87dbf3ecbc2aafae 2024-09-11T07:51:54.636 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:54 smithi110 systemd[1]: Started Ceph mon.a for 41bd1096-7012-11ef-bcea-c7b262605968. 2024-09-11T07:51:55.529 INFO:teuthology.orchestra.run.smithi110.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-11T07:51:55.529 INFO:teuthology.orchestra.run.smithi110.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-11T07:51:55.530 INFO:teuthology.orchestra.run.smithi110.stdout:Creating mgr... 2024-09-11T07:51:55.530 INFO:teuthology.orchestra.run.smithi110.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-11T07:51:55.530 INFO:teuthology.orchestra.run.smithi110.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-11T07:51:55.717 INFO:teuthology.orchestra.run.smithi110.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-41bd1096-7012-11ef-bcea-c7b262605968@mgr.y 2024-09-11T07:51:55.717 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stderr Failed to reset failed state of unit ceph-41bd1096-7012-11ef-bcea-c7b262605968@mgr.y.service: Unit ceph-41bd1096-7012-11ef-bcea-c7b262605968@mgr.y.service not loaded. 2024-09-11T07:51:55.868 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-41bd1096-7012-11ef-bcea-c7b262605968.target.wants/ceph-41bd1096-7012-11ef-bcea-c7b262605968@mgr.y.service → /etc/systemd/system/ceph-41bd1096-7012-11ef-bcea-c7b262605968@.service. 2024-09-11T07:51:56.031 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:51:55 smithi110 systemd[1]: Starting Ceph mgr.y for 41bd1096-7012-11ef-bcea-c7b262605968... 2024-09-11T07:51:56.355 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:56 smithi110 ceph-mon[25392]: from='client.? 172.21.15.110:0/52734405' entity='client.admin' 2024-09-11T07:51:56.355 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:51:56 smithi110 podman[25603]: 2024-09-11 07:51:55.997161475 +0000 UTC m=+0.015420960 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T07:51:56.355 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:51:56 smithi110 podman[25603]: 2024-09-11 07:51:56.135328249 +0000 UTC m=+0.153587723 container create 4cff85987d3da685fed26e768114f2342f49b7dae8329397ade02299eedf8bd0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-11T07:51:56.355 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:51:56 smithi110 podman[25603]: 2024-09-11 07:51:56.354357447 +0000 UTC m=+0.372616922 container init 4cff85987d3da685fed26e768114f2342f49b7dae8329397ade02299eedf8bd0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T07:51:56.496 INFO:teuthology.orchestra.run.smithi110.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T07:51:56.496 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stdout disabled 2024-09-11T07:51:56.505 INFO:teuthology.orchestra.run.smithi110.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T07:51:56.505 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stdout inactive 2024-09-11T07:51:56.505 INFO:teuthology.orchestra.run.smithi110.stdout:firewalld.service is not enabled 2024-09-11T07:51:56.505 INFO:teuthology.orchestra.run.smithi110.stdout:Not possible to enable service . firewalld.service is not available 2024-09-11T07:51:56.516 INFO:teuthology.orchestra.run.smithi110.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T07:51:56.516 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stdout disabled 2024-09-11T07:51:56.525 INFO:teuthology.orchestra.run.smithi110.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T07:51:56.525 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stdout inactive 2024-09-11T07:51:56.525 INFO:teuthology.orchestra.run.smithi110.stdout:firewalld.service is not enabled 2024-09-11T07:51:56.525 INFO:teuthology.orchestra.run.smithi110.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-11T07:51:56.526 INFO:teuthology.orchestra.run.smithi110.stdout:Waiting for mgr to start... 2024-09-11T07:51:56.526 INFO:teuthology.orchestra.run.smithi110.stdout:Waiting for mgr... 2024-09-11T07:51:56.648 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:51:56 smithi110 podman[25603]: 2024-09-11 07:51:56.35742126 +0000 UTC m=+0.375680735 container start 4cff85987d3da685fed26e768114f2342f49b7dae8329397ade02299eedf8bd0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-11T07:51:56.648 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:51:56 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:51:56.468+0000 7ff56cdbbec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T07:51:56.648 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:51:56 smithi110 bash[25603]: 4cff85987d3da685fed26e768114f2342f49b7dae8329397ade02299eedf8bd0 2024-09-11T07:51:56.648 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:51:56 smithi110 systemd[1]: Started Ceph mgr.y for 41bd1096-7012-11ef-bcea-c7b262605968. 2024-09-11T07:51:56.648 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:51:56 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:51:56.534+0000 7ff56cdbbec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T07:51:56.648 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:51:56 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:51:56.602+0000 7ff56cdbbec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T07:51:56.958 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:51:56 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:51:56.726+0000 7ff56cdbbec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T07:51:56.958 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:51:56 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:51:56.867+0000 7ff56cdbbec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T07:51:57.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:51:57 smithi110 ceph-mon[25392]: from='client.? 172.21.15.110:0/372640758' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-11T07:51:57.218 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:51:57 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:51:57.136+0000 7ff56cdbbec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T07:51:57.218 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:51:57 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:51:57.196+0000 7ff56cdbbec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T07:51:57.597 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout 2024-09-11T07:51:57.597 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout { 2024-09-11T07:51:57.597 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "fsid": "41bd1096-7012-11ef-bcea-c7b262605968", 2024-09-11T07:51:57.597 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "health": { 2024-09-11T07:51:57.597 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-11T07:51:57.597 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-11T07:51:57.597 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-11T07:51:57.597 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:51:57.597 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-11T07:51:57.597 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-11T07:51:57.597 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout 0 2024-09-11T07:51:57.598 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout ], 2024-09-11T07:51:57.598 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-11T07:51:57.598 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "a" 2024-09-11T07:51:57.598 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout ], 2024-09-11T07:51:57.598 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-11T07:51:57.598 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-11T07:51:57.598 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T07:51:57.598 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-09-11T07:51:57.598 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-11T07:51:57.598 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:51:57.598 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-11T07:51:57.598 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T07:51:57.598 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-11T07:51:57.598 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-11T07:51:57.598 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-11T07:51:57.599 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-11T07:51:57.599 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-11T07:51:57.599 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-11T07:51:57.599 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:51:57.599 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-11T07:51:57.599 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-11T07:51:57.599 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-11T07:51:57.599 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-11T07:51:57.599 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-11T07:51:57.599 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-11T07:51:57.599 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-11T07:51:57.599 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-11T07:51:57.599 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-11T07:51:57.599 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:51:57.600 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-11T07:51:57.600 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T07:51:57.600 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-11T07:51:57.600 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-11T07:51:57.600 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:51:57.600 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-11T07:51:57.600 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-11T07:51:57.600 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-11T07:51:57.600 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-11T07:51:57.600 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-11T07:51:57.600 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-11T07:51:57.600 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "restful" 2024-09-11T07:51:57.600 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout ], 2024-09-11T07:51:57.600 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T07:51:57.600 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:51:57.601 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-11T07:51:57.601 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T07:51:57.601 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "modified": "2024-09-11T07:51:48.770790+0000", 2024-09-11T07:51:57.601 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T07:51:57.601 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:51:57.601 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-11T07:51:57.601 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout } 2024-09-11T07:51:57.601 INFO:teuthology.orchestra.run.smithi110.stdout:mgr not available, waiting (1/15)... 2024-09-11T07:51:57.718 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:51:57 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:51:57.362+0000 7ff56cdbbec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T07:51:58.468 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:51:58 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:51:58.205+0000 7ff56cdbbec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T07:51:58.468 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:51:58 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:51:58.268+0000 7ff56cdbbec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T07:51:59.090 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:51:58 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:51:58.978+0000 7ff56cdbbec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T07:51:59.090 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:51:59 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:51:59.088+0000 7ff56cdbbec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T07:51:59.357 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:51:59 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:51:59.207+0000 7ff56cdbbec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T07:51:59.357 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:51:59 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:51:59.266+0000 7ff56cdbbec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T07:51:59.699 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:51:59 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:51:59.424+0000 7ff56cdbbec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T07:51:59.699 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:51:59 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:51:59.494+0000 7ff56cdbbec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T07:51:59.699 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:51:59 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:51:59.554+0000 7ff56cdbbec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T07:51:59.699 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:51:59 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:51:59.697+0000 7ff56cdbbec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T07:51:59.968 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:51:59 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:51:59.766+0000 7ff56cdbbec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T07:51:59.969 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:51:59 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:51:59.837+0000 7ff56cdbbec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T07:51:59.969 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:51:59 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:51:59.922+0000 7ff56cdbbec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T07:52:00.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:00 smithi110 ceph-mon[25392]: from='client.? 172.21.15.110:0/2275294879' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-11T07:52:00.468 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:51:59 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:51:59.984+0000 7ff56cdbbec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T07:52:00.468 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:00 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:52:00.046+0000 7ff56cdbbec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T07:52:00.809 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout 2024-09-11T07:52:00.809 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout { 2024-09-11T07:52:00.809 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "fsid": "41bd1096-7012-11ef-bcea-c7b262605968", 2024-09-11T07:52:00.809 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "health": { 2024-09-11T07:52:00.809 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-11T07:52:00.809 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-11T07:52:00.809 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-11T07:52:00.809 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:52:00.809 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-11T07:52:00.810 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-11T07:52:00.810 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout 0 2024-09-11T07:52:00.810 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout ], 2024-09-11T07:52:00.810 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-11T07:52:00.810 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "a" 2024-09-11T07:52:00.810 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout ], 2024-09-11T07:52:00.810 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-11T07:52:00.810 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-11T07:52:00.810 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T07:52:00.810 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-09-11T07:52:00.810 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-11T07:52:00.810 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:52:00.810 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-11T07:52:00.810 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T07:52:00.811 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-11T07:52:00.811 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-11T07:52:00.811 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-11T07:52:00.811 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-11T07:52:00.811 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-11T07:52:00.811 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-11T07:52:00.814 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:52:00.814 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-11T07:52:00.814 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-11T07:52:00.814 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-11T07:52:00.814 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-11T07:52:00.814 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-11T07:52:00.815 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-11T07:52:00.815 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-11T07:52:00.815 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-11T07:52:00.815 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-11T07:52:00.815 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:52:00.815 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-11T07:52:00.815 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T07:52:00.815 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-11T07:52:00.815 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-11T07:52:00.815 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:52:00.815 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-11T07:52:00.815 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-11T07:52:00.815 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-11T07:52:00.815 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-11T07:52:00.816 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-11T07:52:00.816 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-11T07:52:00.816 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "restful" 2024-09-11T07:52:00.816 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout ], 2024-09-11T07:52:00.816 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T07:52:00.816 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:52:00.816 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-11T07:52:00.816 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T07:52:00.816 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "modified": "2024-09-11T07:51:48.770790+0000", 2024-09-11T07:52:00.816 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T07:52:00.816 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:52:00.816 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-11T07:52:00.816 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout } 2024-09-11T07:52:00.816 INFO:teuthology.orchestra.run.smithi110.stdout:mgr not available, waiting (2/15)... 2024-09-11T07:52:01.152 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:00 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:52:00.822+0000 7ff56cdbbec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T07:52:01.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:01 smithi110 ceph-mon[25392]: Activating manager daemon y 2024-09-11T07:52:01.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:01 smithi110 ceph-mon[25392]: mgrmap e2: y(active, starting, since 0.00302177s) 2024-09-11T07:52:01.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:01 smithi110 ceph-mon[25392]: from='mgr.14100 172.21.15.110:0/1681951565' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T07:52:01.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:01 smithi110 ceph-mon[25392]: from='mgr.14100 172.21.15.110:0/1681951565' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T07:52:01.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:01 smithi110 ceph-mon[25392]: from='mgr.14100 172.21.15.110:0/1681951565' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T07:52:01.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:01 smithi110 ceph-mon[25392]: from='mgr.14100 172.21.15.110:0/1681951565' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:52:01.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:01 smithi110 ceph-mon[25392]: from='mgr.14100 172.21.15.110:0/1681951565' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T07:52:01.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:01 smithi110 ceph-mon[25392]: Manager daemon y is now available 2024-09-11T07:52:01.469 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:01 smithi110 ceph-mon[25392]: from='mgr.14100 172.21.15.110:0/1681951565' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T07:52:01.469 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:01 smithi110 ceph-mon[25392]: from='mgr.14100 172.21.15.110:0/1681951565' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T07:52:01.469 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:01 smithi110 ceph-mon[25392]: from='mgr.14100 172.21.15.110:0/1681951565' entity='mgr.y' 2024-09-11T07:52:01.469 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:01 smithi110 ceph-mon[25392]: from='mgr.14100 172.21.15.110:0/1681951565' entity='mgr.y' 2024-09-11T07:52:01.469 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:01 smithi110 ceph-mon[25392]: from='mgr.14100 172.21.15.110:0/1681951565' entity='mgr.y' 2024-09-11T07:52:03.140 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:02 smithi110 ceph-mon[25392]: mgrmap e3: y(active, since 1.00639s) 2024-09-11T07:52:04.048 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout 2024-09-11T07:52:04.048 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout { 2024-09-11T07:52:04.048 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "fsid": "41bd1096-7012-11ef-bcea-c7b262605968", 2024-09-11T07:52:04.048 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "health": { 2024-09-11T07:52:04.048 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-11T07:52:04.049 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-11T07:52:04.049 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-11T07:52:04.049 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:52:04.049 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-11T07:52:04.049 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-11T07:52:04.049 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout 0 2024-09-11T07:52:04.049 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout ], 2024-09-11T07:52:04.049 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-11T07:52:04.049 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "a" 2024-09-11T07:52:04.049 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout ], 2024-09-11T07:52:04.049 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-09-11T07:52:04.049 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-11T07:52:04.049 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T07:52:04.049 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-09-11T07:52:04.050 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-11T07:52:04.050 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:52:04.050 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-11T07:52:04.050 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T07:52:04.050 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-11T07:52:04.050 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-11T07:52:04.050 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-11T07:52:04.050 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-11T07:52:04.050 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-11T07:52:04.050 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-11T07:52:04.053 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:52:04.053 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-11T07:52:04.053 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-11T07:52:04.053 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-11T07:52:04.054 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-11T07:52:04.054 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-11T07:52:04.054 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-11T07:52:04.054 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-11T07:52:04.054 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-11T07:52:04.054 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-11T07:52:04.054 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:52:04.054 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-11T07:52:04.054 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T07:52:04.054 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-11T07:52:04.054 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-11T07:52:04.054 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:52:04.054 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-11T07:52:04.054 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-11T07:52:04.055 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-11T07:52:04.055 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-11T07:52:04.055 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-11T07:52:04.055 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-11T07:52:04.055 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "restful" 2024-09-11T07:52:04.055 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout ], 2024-09-11T07:52:04.055 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T07:52:04.055 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:52:04.055 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-11T07:52:04.055 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T07:52:04.055 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "modified": "2024-09-11T07:51:48.770790+0000", 2024-09-11T07:52:04.055 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T07:52:04.055 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:52:04.055 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-11T07:52:04.055 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout } 2024-09-11T07:52:04.056 INFO:teuthology.orchestra.run.smithi110.stdout:mgr is available 2024-09-11T07:52:04.171 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:03 smithi110 ceph-mon[25392]: mgrmap e4: y(active, since 2s) 2024-09-11T07:52:04.171 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:03 smithi110 ceph-mon[25392]: from='client.? 172.21.15.110:0/3335572075' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-11T07:52:04.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:04 smithi110 ceph-mon[25392]: from='client.? 172.21.15.110:0/1241577209' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-11T07:52:04.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:04 smithi110 ceph-mon[25392]: from='client.? 172.21.15.110:0/1241577209' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-11T07:52:05.312 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout 2024-09-11T07:52:05.312 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout [global] 2024-09-11T07:52:05.312 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout fsid = 41bd1096-7012-11ef-bcea-c7b262605968 2024-09-11T07:52:05.312 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-11T07:52:05.312 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-11T07:52:05.312 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-11T07:52:05.312 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-11T07:52:05.312 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-11T07:52:05.312 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout 2024-09-11T07:52:05.312 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-11T07:52:05.312 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-11T07:52:05.312 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout 2024-09-11T07:52:05.313 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout [osd] 2024-09-11T07:52:05.313 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-11T07:52:05.313 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-11T07:52:05.313 INFO:teuthology.orchestra.run.smithi110.stdout:Enabling cephadm module... 2024-09-11T07:52:06.347 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:05 smithi110 ceph-mon[25392]: from='client.? 172.21.15.110:0/2549270507' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-11T07:52:06.348 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:06 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: ignoring --setuser ceph since I am not root 2024-09-11T07:52:06.348 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:06 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: ignoring --setgroup ceph since I am not root 2024-09-11T07:52:06.348 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:06 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:52:06.119+0000 7faad6ce3ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T07:52:06.348 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:06 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:52:06.173+0000 7faad6ce3ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T07:52:06.348 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:06 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:52:06.237+0000 7faad6ce3ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T07:52:06.598 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:06 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:52:06.364+0000 7faad6ce3ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T07:52:06.598 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:06 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:52:06.501+0000 7faad6ce3ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T07:52:06.892 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:06 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:52:06.762+0000 7faad6ce3ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T07:52:06.892 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:06 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:52:06.824+0000 7faad6ce3ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T07:52:07.178 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:06 smithi110 ceph-mon[25392]: from='client.? 172.21.15.110:0/2549270507' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-11T07:52:07.178 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:06 smithi110 ceph-mon[25392]: mgrmap e5: y(active, since 5s) 2024-09-11T07:52:07.178 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:06 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:52:06.991+0000 7faad6ce3ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T07:52:07.832 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout { 2024-09-11T07:52:07.832 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-11T07:52:07.832 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-11T07:52:07.832 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-11T07:52:07.832 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-11T07:52:07.832 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout } 2024-09-11T07:52:07.832 INFO:teuthology.orchestra.run.smithi110.stdout:Waiting for the mgr to restart... 2024-09-11T07:52:07.832 INFO:teuthology.orchestra.run.smithi110.stdout:Waiting for mgr epoch 5... 2024-09-11T07:52:08.157 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:07 smithi110 ceph-mon[25392]: from='client.? 172.21.15.110:0/28646042' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-11T07:52:08.157 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:07 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:52:07.843+0000 7faad6ce3ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T07:52:08.157 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:07 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:52:07.910+0000 7faad6ce3ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T07:52:08.907 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:08 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:52:08.618+0000 7faad6ce3ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T07:52:08.907 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:08 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:52:08.728+0000 7faad6ce3ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T07:52:08.907 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:08 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:52:08.847+0000 7faad6ce3ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T07:52:08.907 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:08 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:52:08.905+0000 7faad6ce3ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T07:52:09.218 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:09 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:52:09.029+0000 7faad6ce3ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T07:52:09.218 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:09 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:52:09.096+0000 7faad6ce3ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T07:52:09.218 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:09 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:52:09.152+0000 7faad6ce3ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T07:52:09.551 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:09 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:52:09.286+0000 7faad6ce3ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T07:52:09.551 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:09 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:52:09.349+0000 7faad6ce3ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T07:52:09.551 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:09 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:52:09.413+0000 7faad6ce3ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T07:52:09.552 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:09 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:52:09.493+0000 7faad6ce3ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T07:52:09.552 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:09 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:52:09.550+0000 7faad6ce3ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T07:52:09.968 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:09 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:52:09.605+0000 7faad6ce3ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T07:52:10.650 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:10 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:52:10.345+0000 7faad6ce3ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T07:52:10.650 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:10 smithi110 ceph-mon[25392]: Active manager daemon y restarted 2024-09-11T07:52:10.650 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:10 smithi110 ceph-mon[25392]: Activating manager daemon y 2024-09-11T07:52:10.650 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:10 smithi110 ceph-mon[25392]: osdmap e2: 0 total, 0 up, 0 in 2024-09-11T07:52:10.650 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:10 smithi110 ceph-mon[25392]: mgrmap e6: y(active, starting, since 0.00326605s) 2024-09-11T07:52:10.650 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:10 smithi110 ceph-mon[25392]: from='mgr.14118 172.21.15.110:0/119192333' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:52:10.650 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:10 smithi110 ceph-mon[25392]: from='mgr.14118 172.21.15.110:0/119192333' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T07:52:10.651 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:10 smithi110 ceph-mon[25392]: from='mgr.14118 172.21.15.110:0/119192333' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T07:52:10.651 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:10 smithi110 ceph-mon[25392]: from='mgr.14118 172.21.15.110:0/119192333' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T07:52:10.651 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:10 smithi110 ceph-mon[25392]: from='mgr.14118 172.21.15.110:0/119192333' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T07:52:10.651 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:10 smithi110 ceph-mon[25392]: Manager daemon y is now available 2024-09-11T07:52:10.651 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:10 smithi110 ceph-mon[25392]: from='mgr.14118 172.21.15.110:0/119192333' entity='mgr.y' 2024-09-11T07:52:10.651 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:10 smithi110 ceph-mon[25392]: from='mgr.14118 172.21.15.110:0/119192333' entity='mgr.y' 2024-09-11T07:52:10.651 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:10 smithi110 ceph-mon[25392]: from='mgr.14118 172.21.15.110:0/119192333' entity='mgr.y' 2024-09-11T07:52:10.651 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:10 smithi110 ceph-mon[25392]: from='mgr.14118 172.21.15.110:0/119192333' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:52:10.651 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:10 smithi110 ceph-mon[25392]: from='mgr.14118 172.21.15.110:0/119192333' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:52:10.651 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:10 smithi110 ceph-mon[25392]: from='mgr.14118 172.21.15.110:0/119192333' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T07:52:10.652 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:10 smithi110 ceph-mon[25392]: from='mgr.14118 172.21.15.110:0/119192333' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T07:52:11.659 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:11 smithi110 ceph-mon[25392]: Found migration_current of "None". Setting to last migration. 2024-09-11T07:52:11.659 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:11 smithi110 ceph-mon[25392]: mgrmap e7: y(active, since 1.00526s) 2024-09-11T07:52:11.944 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:11 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:52:11.843+0000 7faa890d1640 -1 log_channel(cephadm) log [ERR] : [11/Sep/2024:07:52:11] ENGINE Error in HTTPServer.serve 2024-09-11T07:52:11.944 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:11 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: Traceback (most recent call last): 2024-09-11T07:52:11.944 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:11 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T07:52:11.944 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:11 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: self._connections.run(self.expiration_interval) 2024-09-11T07:52:11.944 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:11 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T07:52:11.944 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:11 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: self._run(expiration_interval) 2024-09-11T07:52:11.945 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:11 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T07:52:11.945 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:11 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T07:52:11.945 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:11 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T07:52:11.945 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:11 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T07:52:11.945 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:11 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T07:52:11.945 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:11 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: s = self.context.wrap_socket( 2024-09-11T07:52:11.945 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:11 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T07:52:11.945 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:11 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: return self.sslsocket_class._create( 2024-09-11T07:52:11.945 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:11 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T07:52:11.945 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:11 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: self.do_handshake() 2024-09-11T07:52:11.945 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:11 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T07:52:11.945 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:11 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: self._sslobj.do_handshake() 2024-09-11T07:52:11.945 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:11 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T07:52:11.945 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:11 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:52:11.947 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout { 2024-09-11T07:52:11.947 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-11T07:52:11.947 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-11T07:52:11.947 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout } 2024-09-11T07:52:11.947 INFO:teuthology.orchestra.run.smithi110.stdout:mgr epoch 5 is available 2024-09-11T07:52:11.947 INFO:teuthology.orchestra.run.smithi110.stdout:Setting orchestrator backend to cephadm... 2024-09-11T07:52:13.066 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:12 smithi110 ceph-mon[25392]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-11T07:52:13.066 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:12 smithi110 ceph-mon[25392]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-11T07:52:13.066 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:12 smithi110 ceph-mon[25392]: [11/Sep/2024:07:52:11] ENGINE Bus STARTING 2024-09-11T07:52:13.066 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:12 smithi110 ceph-mon[25392]: [11/Sep/2024:07:52:11] ENGINE Serving on https://172.21.15.110:7150 2024-09-11T07:52:13.066 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:12 smithi110 ceph-mon[25392]: [11/Sep/2024:07:52:11] ENGINE Bus STARTED 2024-09-11T07:52:13.067 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:12 smithi110 ceph-mon[25392]: [11/Sep/2024:07:52:11] ENGINE Error in HTTPServer.serve 2024-09-11T07:52:13.067 INFO:journalctl@ceph.mon.a.smithi110.stdout: Traceback (most recent call last): 2024-09-11T07:52:13.067 INFO:journalctl@ceph.mon.a.smithi110.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T07:52:13.067 INFO:journalctl@ceph.mon.a.smithi110.stdout: self._connections.run(self.expiration_interval) 2024-09-11T07:52:13.067 INFO:journalctl@ceph.mon.a.smithi110.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T07:52:13.067 INFO:journalctl@ceph.mon.a.smithi110.stdout: self._run(expiration_interval) 2024-09-11T07:52:13.067 INFO:journalctl@ceph.mon.a.smithi110.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T07:52:13.067 INFO:journalctl@ceph.mon.a.smithi110.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T07:52:13.067 INFO:journalctl@ceph.mon.a.smithi110.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T07:52:13.067 INFO:journalctl@ceph.mon.a.smithi110.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T07:52:13.067 INFO:journalctl@ceph.mon.a.smithi110.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T07:52:13.067 INFO:journalctl@ceph.mon.a.smithi110.stdout: s = self.context.wrap_socket( 2024-09-11T07:52:13.067 INFO:journalctl@ceph.mon.a.smithi110.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T07:52:13.067 INFO:journalctl@ceph.mon.a.smithi110.stdout: return self.sslsocket_class._create( 2024-09-11T07:52:13.068 INFO:journalctl@ceph.mon.a.smithi110.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T07:52:13.068 INFO:journalctl@ceph.mon.a.smithi110.stdout: self.do_handshake() 2024-09-11T07:52:13.068 INFO:journalctl@ceph.mon.a.smithi110.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T07:52:13.068 INFO:journalctl@ceph.mon.a.smithi110.stdout: self._sslobj.do_handshake() 2024-09-11T07:52:13.068 INFO:journalctl@ceph.mon.a.smithi110.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T07:52:13.068 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:12 smithi110 ceph-mon[25392]: from='mgr.14118 172.21.15.110:0/119192333' entity='mgr.y' 2024-09-11T07:52:13.068 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:12 smithi110 ceph-mon[25392]: from='mgr.14118 172.21.15.110:0/119192333' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:52:13.068 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:12 smithi110 ceph-mon[25392]: from='mgr.14118 172.21.15.110:0/119192333' entity='mgr.y' 2024-09-11T07:52:13.068 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:12 smithi110 ceph-mon[25392]: from='mgr.14118 172.21.15.110:0/119192333' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:52:13.921 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:13 smithi110 ceph-mon[25392]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:52:13.921 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:13 smithi110 ceph-mon[25392]: mgrmap e8: y(active, since 2s) 2024-09-11T07:52:14.224 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-11T07:52:14.224 INFO:teuthology.orchestra.run.smithi110.stdout:Generating ssh key... 2024-09-11T07:52:15.195 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:14 smithi110 ceph-mon[25392]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:52:15.468 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:15 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: Generating public/private rsa key pair. 2024-09-11T07:52:15.468 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:15 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: Your identification has been saved in /tmp/tmpu93o2rgj/key 2024-09-11T07:52:15.468 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:15 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: Your public key has been saved in /tmp/tmpu93o2rgj/key.pub 2024-09-11T07:52:15.468 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:15 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: The key fingerprint is: 2024-09-11T07:52:15.468 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:15 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: SHA256:w0050i94W2DTJBEPLQxFr1LzrQu/6ovo5Xkc6Yp0dGs ceph-41bd1096-7012-11ef-bcea-c7b262605968 2024-09-11T07:52:15.469 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:15 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: The key's randomart image is: 2024-09-11T07:52:15.469 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:15 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: +---[RSA 3072]----+ 2024-09-11T07:52:15.469 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:15 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: | .=B+. | 2024-09-11T07:52:15.469 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:15 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: | .oO. | 2024-09-11T07:52:15.469 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:15 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: | . @.+ | 2024-09-11T07:52:15.469 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:15 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: | . B O . | 2024-09-11T07:52:15.469 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:15 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: | S *.+ . | 2024-09-11T07:52:15.469 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:15 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: | . =o= . | 2024-09-11T07:52:15.469 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:15 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: | . ooE.. | 2024-09-11T07:52:15.469 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:15 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: | . * ++o . | 2024-09-11T07:52:15.469 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:15 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: | .+ =++o+. | 2024-09-11T07:52:15.469 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:15 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: +----[SHA256]-----+ 2024-09-11T07:52:16.114 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:15 smithi110 ceph-mon[25392]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:52:16.114 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:15 smithi110 ceph-mon[25392]: Generating ssh key... 2024-09-11T07:52:16.114 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:15 smithi110 ceph-mon[25392]: from='mgr.14118 172.21.15.110:0/119192333' entity='mgr.y' 2024-09-11T07:52:16.114 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:15 smithi110 ceph-mon[25392]: from='mgr.14118 172.21.15.110:0/119192333' entity='mgr.y' 2024-09-11T07:52:16.903 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCtb2idwJAse4LcM8K8OGqmKG5DOhnpSW6de+iRyLnpLr4zrDdqRZBCkqoNC88dWujmv0pgOf1C3DN4z20rAQQSaFlDBbjFQjvXSHQ8w4OmMMAR7kB4nlyWYG6ZVP8srR7lEVXKjBx80kKWyoDCqUYIC5RdhKCUYAbJfRSfTLdtXRDOitVG+Qe/jSi+JhzyYsTh9XU43fEfCI2QwtDFJzM3ITc3SwYjv1sBrcWUcFZLgoGJdVFXEYm7jION4qrXdBxg0D890SXyeJzKLDhJasCMBQvOku8mN+/ayqwBRn1LsPznu0LAUeXAC32ELP8FMzMKCrL/i8CwuRaXbQIk6yqwDZxB/WMCdRsl5B4nWecLamdtsz/OOHZVjucnPhFaGfHvnSkFW3/rCkb5OhkDoUk75QIexT/1w+Jt37+7bmGFguBqmW8vu2ZoI+BjHIti82oF+URnTApcE4vXkC1CDyHx2S7R5QScvfhmW2UfofepX/1mafLz/pzVHTx0j9KefyE= ceph-41bd1096-7012-11ef-bcea-c7b262605968 2024-09-11T07:52:16.903 INFO:teuthology.orchestra.run.smithi110.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-11T07:52:16.904 INFO:teuthology.orchestra.run.smithi110.stdout:Adding key to root@localhost authorized_keys... 2024-09-11T07:52:16.910 INFO:teuthology.orchestra.run.smithi110.stdout:Adding host smithi110... 2024-09-11T07:52:17.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:17 smithi110 ceph-mon[25392]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:52:18.260 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:18 smithi110 ceph-mon[25392]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi110", "addr": "172.21.15.110", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:52:19.202 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:19 smithi110 ceph-mon[25392]: Deploying cephadm binary to smithi110 2024-09-11T07:52:20.711 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout Added host 'smithi110' with addr '172.21.15.110' 2024-09-11T07:52:20.711 INFO:teuthology.orchestra.run.smithi110.stdout:Deploying unmanaged mon service... 2024-09-11T07:52:21.255 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:21 smithi110 ceph-mon[25392]: from='mgr.14118 172.21.15.110:0/119192333' entity='mgr.y' 2024-09-11T07:52:21.255 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:21 smithi110 ceph-mon[25392]: Added host smithi110 2024-09-11T07:52:21.255 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:21 smithi110 ceph-mon[25392]: from='mgr.14118 172.21.15.110:0/119192333' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:52:22.490 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-11T07:52:22.490 INFO:teuthology.orchestra.run.smithi110.stdout:Deploying unmanaged mgr service... 2024-09-11T07:52:22.698 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:22 smithi110 ceph-mon[25392]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:52:22.698 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:22 smithi110 ceph-mon[25392]: Saving service mon spec with placement count:5 2024-09-11T07:52:22.698 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:22 smithi110 ceph-mon[25392]: from='mgr.14118 172.21.15.110:0/119192333' entity='mgr.y' 2024-09-11T07:52:24.115 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-11T07:52:24.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:24 smithi110 ceph-mon[25392]: from='mgr.14118 172.21.15.110:0/119192333' entity='mgr.y' 2024-09-11T07:52:24.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:24 smithi110 ceph-mon[25392]: from='mgr.14118 172.21.15.110:0/119192333' entity='mgr.y' 2024-09-11T07:52:24.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:24 smithi110 ceph-mon[25392]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:52:24.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:24 smithi110 ceph-mon[25392]: Saving service mgr spec with placement count:2 2024-09-11T07:52:24.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:24 smithi110 ceph-mon[25392]: from='mgr.14118 172.21.15.110:0/119192333' entity='mgr.y' 2024-09-11T07:52:25.880 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:25 smithi110 ceph-mon[25392]: from='client.? 172.21.15.110:0/2530407482' entity='client.admin' 2024-09-11T07:52:25.880 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:25 smithi110 ceph-mon[25392]: from='mgr.14118 172.21.15.110:0/119192333' entity='mgr.y' 2024-09-11T07:52:25.880 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:25 smithi110 ceph-mon[25392]: from='mgr.14118 172.21.15.110:0/119192333' entity='mgr.y' 2024-09-11T07:52:25.881 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:25 smithi110 ceph-mon[25392]: from='mgr.14118 172.21.15.110:0/119192333' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi110", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:52:25.881 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:25 smithi110 ceph-mon[25392]: from='mgr.14118 172.21.15.110:0/119192333' entity='mgr.y' 2024-09-11T07:52:25.881 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:25 smithi110 ceph-mon[25392]: from='mgr.14118 172.21.15.110:0/119192333' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi110", "caps": []}]: dispatch 2024-09-11T07:52:25.881 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:25 smithi110 ceph-mon[25392]: from='mgr.14118 172.21.15.110:0/119192333' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.agent.smithi110", "caps": []}]': finished 2024-09-11T07:52:26.794 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:26 smithi110 ceph-mon[25392]: Deploying daemon agent.smithi110 on smithi110 2024-09-11T07:52:26.794 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:26 smithi110 ceph-mon[25392]: from='client.? 172.21.15.110:0/2010208644' entity='client.admin' 2024-09-11T07:52:27.089 INFO:teuthology.orchestra.run.smithi110.stdout:Enabling the dashboard module... 2024-09-11T07:52:27.964 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:27 smithi110 ceph-mon[25392]: from='client.? 172.21.15.110:0/671900623' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-11T07:52:27.965 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:27 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: ignoring --setuser ceph since I am not root 2024-09-11T07:52:27.965 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:27 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: ignoring --setgroup ceph since I am not root 2024-09-11T07:52:27.965 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:27 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:52:27.842+0000 7f23a110bec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T07:52:27.965 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:27 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:52:27.897+0000 7f23a110bec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T07:52:27.965 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:27 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:52:27.961+0000 7f23a110bec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T07:52:28.218 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:28 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:52:28.086+0000 7f23a110bec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T07:52:28.486 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:28 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:52:28.223+0000 7f23a110bec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T07:52:28.486 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:28 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:52:28.485+0000 7f23a110bec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T07:52:28.812 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:28 smithi110 ceph-mon[25392]: from='client.? 172.21.15.110:0/671900623' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-11T07:52:28.813 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:28 smithi110 ceph-mon[25392]: mgrmap e9: y(active, since 17s) 2024-09-11T07:52:28.813 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:28 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:52:28.545+0000 7f23a110bec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T07:52:28.813 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:28 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:52:28.713+0000 7f23a110bec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T07:52:29.595 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout { 2024-09-11T07:52:29.595 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-11T07:52:29.595 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-11T07:52:29.595 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-11T07:52:29.596 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-11T07:52:29.596 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout } 2024-09-11T07:52:29.596 INFO:teuthology.orchestra.run.smithi110.stdout:Waiting for the mgr to restart... 2024-09-11T07:52:29.596 INFO:teuthology.orchestra.run.smithi110.stdout:Waiting for mgr epoch 9... 2024-09-11T07:52:29.841 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:29 smithi110 ceph-mon[25392]: from='client.? 172.21.15.110:0/1543058584' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-11T07:52:29.841 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:29 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:52:29.582+0000 7f23a110bec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T07:52:29.842 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:29 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:52:29.651+0000 7f23a110bec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T07:52:30.657 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:30 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:52:30.369+0000 7f23a110bec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T07:52:30.658 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:30 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:52:30.478+0000 7f23a110bec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T07:52:30.658 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:30 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:52:30.597+0000 7f23a110bec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T07:52:30.658 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:30 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:52:30.656+0000 7f23a110bec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T07:52:30.968 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:30 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:52:30.781+0000 7f23a110bec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T07:52:30.968 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:30 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:52:30.846+0000 7f23a110bec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T07:52:30.968 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:30 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:52:30.904+0000 7f23a110bec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T07:52:31.302 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:31 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:52:31.038+0000 7f23a110bec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T07:52:31.302 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:31 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:52:31.101+0000 7f23a110bec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T07:52:31.302 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:31 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:52:31.164+0000 7f23a110bec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T07:52:31.302 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:31 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:52:31.244+0000 7f23a110bec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T07:52:31.302 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:31 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:52:31.300+0000 7f23a110bec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T07:52:31.718 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:31 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:52:31.357+0000 7f23a110bec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T07:52:32.163 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:32 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:52:32.109+0000 7f23a110bec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T07:52:32.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:32 smithi110 ceph-mon[25392]: Active manager daemon y restarted 2024-09-11T07:52:32.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:32 smithi110 ceph-mon[25392]: Activating manager daemon y 2024-09-11T07:52:32.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:32 smithi110 ceph-mon[25392]: osdmap e3: 0 total, 0 up, 0 in 2024-09-11T07:52:32.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:32 smithi110 ceph-mon[25392]: mgrmap e10: y(active, starting, since 0.00307646s) 2024-09-11T07:52:32.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:32 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:52:32.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:32 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T07:52:32.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:32 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T07:52:32.469 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:32 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T07:52:32.469 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:32 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T07:52:32.469 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:32 smithi110 ceph-mon[25392]: Manager daemon y is now available 2024-09-11T07:52:32.469 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:32 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:52:32.469 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:32 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:52:32.469 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:32 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T07:52:32.469 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:32 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T07:52:33.962 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout { 2024-09-11T07:52:33.963 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-11T07:52:33.963 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-11T07:52:33.963 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout } 2024-09-11T07:52:33.963 INFO:teuthology.orchestra.run.smithi110.stdout:mgr epoch 9 is available 2024-09-11T07:52:33.963 INFO:teuthology.orchestra.run.smithi110.stdout:Generating a dashboard self-signed certificate... 2024-09-11T07:52:34.212 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:34 smithi110 ceph-mon[25392]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-11T07:52:34.212 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:34 smithi110 ceph-mon[25392]: mgrmap e11: y(active, since 1.00549s) 2024-09-11T07:52:34.212 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:34 smithi110 ceph-mon[25392]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-11T07:52:34.212 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:34 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:52:34.212 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:34 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi110", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:52:34.212 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:34 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:52:34.212 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:34 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi110", "caps": []}]: dispatch 2024-09-11T07:52:34.469 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:34 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:52:34.209+0000 7f2351c68640 -1 log_channel(cephadm) log [ERR] : [11/Sep/2024:07:52:34] ENGINE Error in HTTPServer.serve 2024-09-11T07:52:34.469 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:34 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: Traceback (most recent call last): 2024-09-11T07:52:34.469 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:34 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T07:52:34.469 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:34 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: self._connections.run(self.expiration_interval) 2024-09-11T07:52:34.469 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:34 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T07:52:34.469 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:34 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: self._run(expiration_interval) 2024-09-11T07:52:34.469 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:34 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T07:52:34.469 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:34 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T07:52:34.470 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:34 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T07:52:34.470 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:34 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T07:52:34.470 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:34 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T07:52:34.470 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:34 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: s = self.context.wrap_socket( 2024-09-11T07:52:34.470 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:34 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T07:52:34.470 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:34 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: return self.sslsocket_class._create( 2024-09-11T07:52:34.470 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:34 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T07:52:34.470 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:34 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: self.do_handshake() 2024-09-11T07:52:34.470 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:34 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T07:52:34.470 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:34 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: self._sslobj.do_handshake() 2024-09-11T07:52:34.470 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:34 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T07:52:34.470 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:52:34 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:52:35.101 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-11T07:52:35.101 INFO:teuthology.orchestra.run.smithi110.stdout:Creating initial admin user... 2024-09-11T07:52:35.358 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:35 smithi110 ceph-mon[25392]: [11/Sep/2024:07:52:34] ENGINE Bus STARTING 2024-09-11T07:52:35.358 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:35 smithi110 ceph-mon[25392]: Deploying daemon agent.smithi110 on smithi110 2024-09-11T07:52:35.358 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:35 smithi110 ceph-mon[25392]: [11/Sep/2024:07:52:34] ENGINE Serving on https://172.21.15.110:7150 2024-09-11T07:52:35.358 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:35 smithi110 ceph-mon[25392]: [11/Sep/2024:07:52:34] ENGINE Bus STARTED 2024-09-11T07:52:35.358 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:35 smithi110 ceph-mon[25392]: [11/Sep/2024:07:52:34] ENGINE Error in HTTPServer.serve 2024-09-11T07:52:35.358 INFO:journalctl@ceph.mon.a.smithi110.stdout: Traceback (most recent call last): 2024-09-11T07:52:35.358 INFO:journalctl@ceph.mon.a.smithi110.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T07:52:35.358 INFO:journalctl@ceph.mon.a.smithi110.stdout: self._connections.run(self.expiration_interval) 2024-09-11T07:52:35.358 INFO:journalctl@ceph.mon.a.smithi110.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T07:52:35.358 INFO:journalctl@ceph.mon.a.smithi110.stdout: self._run(expiration_interval) 2024-09-11T07:52:35.358 INFO:journalctl@ceph.mon.a.smithi110.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T07:52:35.358 INFO:journalctl@ceph.mon.a.smithi110.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T07:52:35.359 INFO:journalctl@ceph.mon.a.smithi110.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T07:52:35.359 INFO:journalctl@ceph.mon.a.smithi110.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T07:52:35.359 INFO:journalctl@ceph.mon.a.smithi110.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T07:52:35.359 INFO:journalctl@ceph.mon.a.smithi110.stdout: s = self.context.wrap_socket( 2024-09-11T07:52:35.359 INFO:journalctl@ceph.mon.a.smithi110.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T07:52:35.359 INFO:journalctl@ceph.mon.a.smithi110.stdout: return self.sslsocket_class._create( 2024-09-11T07:52:35.359 INFO:journalctl@ceph.mon.a.smithi110.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T07:52:35.359 INFO:journalctl@ceph.mon.a.smithi110.stdout: self.do_handshake() 2024-09-11T07:52:35.359 INFO:journalctl@ceph.mon.a.smithi110.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T07:52:35.359 INFO:journalctl@ceph.mon.a.smithi110.stdout: self._sslobj.do_handshake() 2024-09-11T07:52:35.359 INFO:journalctl@ceph.mon.a.smithi110.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T07:52:35.359 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:35 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:52:35.359 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:35 smithi110 ceph-mon[25392]: mgrmap e12: y(active, since 2s) 2024-09-11T07:52:35.359 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:35 smithi110 ceph-mon[25392]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:52:35.360 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:35 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:52:35.360 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:35 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:52:36.701 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$rgNEUeQijpZj0dVvP8xgteXPKotXnvzZMVhAyuiAkhj/0GqF6mC26", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726041156, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-11T07:52:36.701 INFO:teuthology.orchestra.run.smithi110.stdout:Fetching dashboard port number... 2024-09-11T07:52:37.239 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:37 smithi110 ceph-mon[25392]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:52:37.239 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:37 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:52:37.805 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout 8443 2024-09-11T07:52:37.815 INFO:teuthology.orchestra.run.smithi110.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T07:52:37.815 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stdout disabled 2024-09-11T07:52:37.822 INFO:teuthology.orchestra.run.smithi110.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T07:52:37.822 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stdout inactive 2024-09-11T07:52:37.822 INFO:teuthology.orchestra.run.smithi110.stdout:firewalld.service is not enabled 2024-09-11T07:52:37.823 INFO:teuthology.orchestra.run.smithi110.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-11T07:52:37.825 INFO:teuthology.orchestra.run.smithi110.stdout:Ceph Dashboard is now available at: 2024-09-11T07:52:37.825 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T07:52:37.825 INFO:teuthology.orchestra.run.smithi110.stdout: URL: https://smithi110.front.sepia.ceph.com:8443/ 2024-09-11T07:52:37.825 INFO:teuthology.orchestra.run.smithi110.stdout: User: admin 2024-09-11T07:52:37.825 INFO:teuthology.orchestra.run.smithi110.stdout: Password: 5gk0282obw 2024-09-11T07:52:37.825 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T07:52:37.826 INFO:teuthology.orchestra.run.smithi110.stdout:Saving cluster configuration to /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/config directory 2024-09-11T07:52:37.826 INFO:teuthology.orchestra.run.smithi110.stdout:Enabling autotune for osd_memory_target 2024-09-11T07:52:38.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:38 smithi110 ceph-mon[25392]: from='client.? 172.21.15.110:0/2802988921' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-11T07:52:40.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:40 smithi110 ceph-mon[25392]: mgrmap e13: y(active, since 6s) 2024-09-11T07:52:40.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:40 smithi110 ceph-mon[25392]: from='client.? 172.21.15.110:0/3185331542' entity='client.admin' 2024-09-11T07:52:40.510 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-11T07:52:40.510 INFO:teuthology.orchestra.run.smithi110.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-11T07:52:40.510 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T07:52:40.510 INFO:teuthology.orchestra.run.smithi110.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-11T07:52:40.510 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T07:52:40.510 INFO:teuthology.orchestra.run.smithi110.stdout:Or, if you are only running a single cluster on this host: 2024-09-11T07:52:40.510 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T07:52:40.510 INFO:teuthology.orchestra.run.smithi110.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-11T07:52:40.510 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T07:52:40.511 INFO:teuthology.orchestra.run.smithi110.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-11T07:52:40.511 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T07:52:40.511 INFO:teuthology.orchestra.run.smithi110.stdout: ceph telemetry on 2024-09-11T07:52:40.511 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T07:52:40.511 INFO:teuthology.orchestra.run.smithi110.stdout:For more information see: 2024-09-11T07:52:40.511 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T07:52:40.511 INFO:teuthology.orchestra.run.smithi110.stdout: https://docs.ceph.com/docs/master/mgr/telemetry/ 2024-09-11T07:52:40.511 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T07:52:40.511 INFO:teuthology.orchestra.run.smithi110.stdout:Bootstrap complete. 2024-09-11T07:52:40.554 INFO:tasks.cephadm:Fetching config... 2024-09-11T07:52:40.554 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-11T07:52:40.554 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-11T07:52:40.572 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-11T07:52:40.573 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-11T07:52:40.573 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-11T07:52:40.632 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-11T07:52:40.632 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-11T07:52:40.632 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-11T07:52:40.743 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-11T07:52:40.743 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-11T07:52:40.743 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-11T07:52:40.800 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-11T07:52:40.800 DEBUG:teuthology.orchestra.run.smithi110:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCtb2idwJAse4LcM8K8OGqmKG5DOhnpSW6de+iRyLnpLr4zrDdqRZBCkqoNC88dWujmv0pgOf1C3DN4z20rAQQSaFlDBbjFQjvXSHQ8w4OmMMAR7kB4nlyWYG6ZVP8srR7lEVXKjBx80kKWyoDCqUYIC5RdhKCUYAbJfRSfTLdtXRDOitVG+Qe/jSi+JhzyYsTh9XU43fEfCI2QwtDFJzM3ITc3SwYjv1sBrcWUcFZLgoGJdVFXEYm7jION4qrXdBxg0D890SXyeJzKLDhJasCMBQvOku8mN+/ayqwBRn1LsPznu0LAUeXAC32ELP8FMzMKCrL/i8CwuRaXbQIk6yqwDZxB/WMCdRsl5B4nWecLamdtsz/OOHZVjucnPhFaGfHvnSkFW3/rCkb5OhkDoUk75QIexT/1w+Jt37+7bmGFguBqmW8vu2ZoI+BjHIti82oF+URnTApcE4vXkC1CDyHx2S7R5QScvfhmW2UfofepX/1mafLz/pzVHTx0j9KefyE= ceph-41bd1096-7012-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-11T07:52:40.963 INFO:teuthology.orchestra.run.smithi110.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCtb2idwJAse4LcM8K8OGqmKG5DOhnpSW6de+iRyLnpLr4zrDdqRZBCkqoNC88dWujmv0pgOf1C3DN4z20rAQQSaFlDBbjFQjvXSHQ8w4OmMMAR7kB4nlyWYG6ZVP8srR7lEVXKjBx80kKWyoDCqUYIC5RdhKCUYAbJfRSfTLdtXRDOitVG+Qe/jSi+JhzyYsTh9XU43fEfCI2QwtDFJzM3ITc3SwYjv1sBrcWUcFZLgoGJdVFXEYm7jION4qrXdBxg0D890SXyeJzKLDhJasCMBQvOku8mN+/ayqwBRn1LsPznu0LAUeXAC32ELP8FMzMKCrL/i8CwuRaXbQIk6yqwDZxB/WMCdRsl5B4nWecLamdtsz/OOHZVjucnPhFaGfHvnSkFW3/rCkb5OhkDoUk75QIexT/1w+Jt37+7bmGFguBqmW8vu2ZoI+BjHIti82oF+URnTApcE4vXkC1CDyHx2S7R5QScvfhmW2UfofepX/1mafLz/pzVHTx0j9KefyE= ceph-41bd1096-7012-11ef-bcea-c7b262605968 2024-09-11T07:52:40.975 DEBUG:teuthology.orchestra.run.smithi139:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCtb2idwJAse4LcM8K8OGqmKG5DOhnpSW6de+iRyLnpLr4zrDdqRZBCkqoNC88dWujmv0pgOf1C3DN4z20rAQQSaFlDBbjFQjvXSHQ8w4OmMMAR7kB4nlyWYG6ZVP8srR7lEVXKjBx80kKWyoDCqUYIC5RdhKCUYAbJfRSfTLdtXRDOitVG+Qe/jSi+JhzyYsTh9XU43fEfCI2QwtDFJzM3ITc3SwYjv1sBrcWUcFZLgoGJdVFXEYm7jION4qrXdBxg0D890SXyeJzKLDhJasCMBQvOku8mN+/ayqwBRn1LsPznu0LAUeXAC32ELP8FMzMKCrL/i8CwuRaXbQIk6yqwDZxB/WMCdRsl5B4nWecLamdtsz/OOHZVjucnPhFaGfHvnSkFW3/rCkb5OhkDoUk75QIexT/1w+Jt37+7bmGFguBqmW8vu2ZoI+BjHIti82oF+URnTApcE4vXkC1CDyHx2S7R5QScvfhmW2UfofepX/1mafLz/pzVHTx0j9KefyE= ceph-41bd1096-7012-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-11T07:52:41.218 INFO:teuthology.orchestra.run.smithi139.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCtb2idwJAse4LcM8K8OGqmKG5DOhnpSW6de+iRyLnpLr4zrDdqRZBCkqoNC88dWujmv0pgOf1C3DN4z20rAQQSaFlDBbjFQjvXSHQ8w4OmMMAR7kB4nlyWYG6ZVP8srR7lEVXKjBx80kKWyoDCqUYIC5RdhKCUYAbJfRSfTLdtXRDOitVG+Qe/jSi+JhzyYsTh9XU43fEfCI2QwtDFJzM3ITc3SwYjv1sBrcWUcFZLgoGJdVFXEYm7jION4qrXdBxg0D890SXyeJzKLDhJasCMBQvOku8mN+/ayqwBRn1LsPznu0LAUeXAC32ELP8FMzMKCrL/i8CwuRaXbQIk6yqwDZxB/WMCdRsl5B4nWecLamdtsz/OOHZVjucnPhFaGfHvnSkFW3/rCkb5OhkDoUk75QIexT/1w+Jt37+7bmGFguBqmW8vu2ZoI+BjHIti82oF+URnTApcE4vXkC1CDyHx2S7R5QScvfhmW2UfofepX/1mafLz/pzVHTx0j9KefyE= ceph-41bd1096-7012-11ef-bcea-c7b262605968 2024-09-11T07:52:41.239 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-11T07:52:41.631 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:52:42.590 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:42 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:52:42.590 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:42 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:52:42.590 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:42 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:52:42.590 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:42 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:52:42.590 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:42 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:52:42.590 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:42 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:52:42.590 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:42 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:52:42.591 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:42 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:52:42.591 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:42 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:52:44.030 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-11T07:52:44.030 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-11T07:52:44.606 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:44 smithi110 ceph-mon[25392]: from='client.? 172.21.15.110:0/1402825235' entity='client.admin' 2024-09-11T07:52:44.606 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:44 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:52:44.606 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:44 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:52:45.025 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:52:47.839 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi139 2024-09-11T07:52:47.839 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-11T07:52:47.839 DEBUG:teuthology.orchestra.run.smithi139:> dd of=/etc/ceph/ceph.conf 2024-09-11T07:52:47.859 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-11T07:52:47.860 DEBUG:teuthology.orchestra.run.smithi139:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T07:52:47.918 INFO:tasks.cephadm:Adding host smithi139 to orchestrator... 2024-09-11T07:52:47.919 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- ceph orch host add smithi139 2024-09-11T07:52:48.157 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:47 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:52:48.157 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:47 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:52:48.157 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:47 smithi110 ceph-mon[25392]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:52:48.157 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:47 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:52:48.157 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:47 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:52:48.157 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:47 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:52:48.157 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:47 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:52:48.157 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:47 smithi110 ceph-mon[25392]: Updating smithi110:/etc/ceph/ceph.conf 2024-09-11T07:52:48.157 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:47 smithi110 ceph-mon[25392]: Updating smithi110:/var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:52:48.157 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:47 smithi110 ceph-mon[25392]: Updating smithi110:/etc/ceph/ceph.client.admin.keyring 2024-09-11T07:52:48.158 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:47 smithi110 ceph-mon[25392]: Updating smithi110:/var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T07:52:48.158 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:47 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:52:48.158 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:47 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:52:48.238 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:52:50.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:50 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:52:50.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:50 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:52:51.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:51 smithi110 ceph-mon[25392]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi139", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:52:52.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:52 smithi110 ceph-mon[25392]: Deploying cephadm binary to smithi139 2024-09-11T07:52:53.054 INFO:teuthology.orchestra.run.smithi110.stdout:Added host 'smithi139' with addr '172.21.15.139' 2024-09-11T07:52:53.664 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:53 smithi110 ceph-mon[25392]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:52:53.664 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:53 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:52:53.664 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:53 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:52:53.664 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:53 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:52:53.688 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-11T07:52:53.930 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:52:54.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:54 smithi110 ceph-mon[25392]: Added host smithi139 2024-09-11T07:52:54.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:54 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:52:54.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:54 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:52:54.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:54 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi139", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:52:54.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:54 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:52:54.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:54 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:52:54.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:54 smithi110 ceph-mon[25392]: Updating smithi139:/etc/ceph/ceph.conf 2024-09-11T07:52:54.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:54 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:52:54.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:54 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:52:54.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:54 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi139", "caps": []}]: dispatch 2024-09-11T07:52:54.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:54 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.agent.smithi139", "caps": []}]': finished 2024-09-11T07:52:55.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:55 smithi110 ceph-mon[25392]: Updating smithi139:/var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:52:55.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:55 smithi110 ceph-mon[25392]: Updating smithi139:/etc/ceph/ceph.client.admin.keyring 2024-09-11T07:52:55.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:55 smithi110 ceph-mon[25392]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:52:55.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:55 smithi110 ceph-mon[25392]: Updating smithi139:/var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T07:52:55.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:55 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:52:55.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:55 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:52:55.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:55 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:52:55.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:55 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:52:55.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:55 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:52:55.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:55 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:52:55.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:55 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:52:55.736 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T07:52:55.736 INFO:teuthology.orchestra.run.smithi110.stdout:[{"addr": "172.21.15.110", "hostname": "smithi110", "labels": [], "status": ""}, {"addr": "172.21.15.139", "hostname": "smithi139", "labels": [], "status": ""}] 2024-09-11T07:52:56.351 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-11T07:52:56.351 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- ceph osd crush tunables default 2024-09-11T07:52:56.584 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:52:56.863 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:56 smithi110 ceph-mon[25392]: Deploying daemon agent.smithi139 on smithi139 2024-09-11T07:52:56.863 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:56 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:52:56.864 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:56 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:52:56.864 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:56 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:52:56.864 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:56 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:52:56.864 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:56 smithi110 ceph-mon[25392]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:52:56.864 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:56 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:52:56.864 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:56 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:52:56.864 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:56 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:52:57.836 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:57 smithi110 ceph-mon[25392]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:52:57.836 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:57 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:52:58.649 INFO:teuthology.orchestra.run.smithi110.stderr:adjusted tunables profile to default 2024-09-11T07:52:58.955 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:58 smithi110 ceph-mon[25392]: from='client.? 172.21.15.110:0/2982156807' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-11T07:52:59.240 INFO:tasks.cephadm:Adding mon.a on smithi110 2024-09-11T07:52:59.241 INFO:tasks.cephadm:Adding mon.c on smithi110 2024-09-11T07:52:59.241 INFO:tasks.cephadm:Adding mon.b on smithi139 2024-09-11T07:52:59.241 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- ceph orch apply mon '3;smithi110:172.21.15.110=a;smithi110:[v2:172.21.15.110:3301,v1:172.21.15.110:6790]=c;smithi139:172.21.15.139=b' 2024-09-11T07:52:59.473 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:52:59.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:59 smithi110 ceph-mon[25392]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:52:59.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:59 smithi110 ceph-mon[25392]: from='client.? 172.21.15.110:0/2982156807' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-11T07:52:59.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:59 smithi110 ceph-mon[25392]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T07:52:59.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:59 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:52:59.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:52:59 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:01.020 INFO:teuthology.orchestra.run.smithi139.stdout:Scheduled mon update... 2024-09-11T07:53:01.594 DEBUG:teuthology.orchestra.run.smithi110:mon.c> sudo journalctl -f -n 0 -u ceph-41bd1096-7012-11ef-bcea-c7b262605968@mon.c.service 2024-09-11T07:53:01.597 DEBUG:teuthology.orchestra.run.smithi139:mon.b> sudo journalctl -f -n 0 -u ceph-41bd1096-7012-11ef-bcea-c7b262605968@mon.b.service 2024-09-11T07:53:01.599 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T07:53:01.599 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T07:53:01.944 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:01 smithi110 ceph-mon[25392]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:01.944 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:01 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:01.944 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:01 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:53:01.944 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:01 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:53:01.944 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:01 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:53:01.944 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:01 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:01.945 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:01 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T07:53:01.945 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:01 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:53:02.875 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:02 smithi110 ceph-mon[25392]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi110:172.21.15.110=a;smithi110:[v2:172.21.15.110:3301,v1:172.21.15.110:6790]=c;smithi139:172.21.15.139=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:53:02.875 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:02 smithi110 ceph-mon[25392]: Saving service mon spec with placement smithi110:172.21.15.110=a;smithi110:[v2:172.21.15.110:3301,v1:172.21.15.110:6790]=c;smithi139:172.21.15.139=b;count:3 2024-09-11T07:53:02.875 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:02 smithi110 ceph-mon[25392]: Deploying daemon mon.b on smithi139 2024-09-11T07:53:02.875 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:02 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:02.875 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:02 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:02.875 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:02 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:03.441 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:53:03.698 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:03 smithi110 ceph-mon[25392]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:05.782 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-11T07:53:05.782 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":1,"fsid":"41bd1096-7012-11ef-bcea-c7b262605968","modified":"2024-09-11T07:51:39.629440Z","created":"2024-09-11T07:51:39.629440Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:3300","nonce":0},{"type":"v1","addr":"172.21.15.110:6789","nonce":0}]},"addr":"172.21.15.110:6789/0","public_addr":"172.21.15.110:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-11T07:53:05.783 INFO:teuthology.orchestra.run.smithi139.stderr:dumped monmap epoch 1 2024-09-11T07:53:05.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:05 smithi110 ceph-mon[25392]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:06.454 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 podman[27275]: 2024-09-11 07:53:06.434367239 +0000 UTC m=+0.852929107 container start 4372b142090407d319454577968ec4111fd04b600ecedf8fd0bdf2cc2aec6f80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-mon-b, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True) 2024-09-11T07:53:06.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: set uid:gid to 167:167 (ceph:ceph) 2024-09-11T07:53:06.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable), process ceph-mon, pid 2 2024-09-11T07:53:06.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: pidfile_write: ignore empty --pid-file 2024-09-11T07:53:06.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: load: jerasure load: lrc 2024-09-11T07:53:06.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: RocksDB version: 6.15.5 2024-09-11T07:53:06.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-09-11T07:53:06.750 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Compile date Sep 10 2024 2024-09-11T07:53:06.750 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: DB SUMMARY 2024-09-11T07:53:06.750 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: DB Session ID: 0YLXSRV0J7B4MNH02AGW 2024-09-11T07:53:06.750 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: CURRENT file: CURRENT 2024-09-11T07:53:06.750 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: IDENTITY file: IDENTITY 2024-09-11T07:53:06.750 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: MANIFEST file: MANIFEST-000003 size: 57 Bytes 2024-09-11T07:53:06.750 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-09-11T07:53:06.750 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 511 ; 2024-09-11T07:53:06.750 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.error_if_exists: 0 2024-09-11T07:53:06.750 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.create_if_missing: 0 2024-09-11T07:53:06.750 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.paranoid_checks: 1 2024-09-11T07:53:06.750 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-11T07:53:06.750 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.env: 0x559af772d440 2024-09-11T07:53:06.751 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.fs: Posix File System 2024-09-11T07:53:06.751 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.info_log: 0x559af88aaec0 2024-09-11T07:53:06.751 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.max_file_opening_threads: 16 2024-09-11T07:53:06.751 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.statistics: (nil) 2024-09-11T07:53:06.751 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.use_fsync: 0 2024-09-11T07:53:06.751 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.max_log_file_size: 0 2024-09-11T07:53:06.751 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-11T07:53:06.751 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-11T07:53:06.751 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.keep_log_file_num: 1000 2024-09-11T07:53:06.751 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.recycle_log_file_num: 0 2024-09-11T07:53:06.751 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.allow_fallocate: 1 2024-09-11T07:53:06.751 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.allow_mmap_reads: 0 2024-09-11T07:53:06.751 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.allow_mmap_writes: 0 2024-09-11T07:53:06.751 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.use_direct_reads: 0 2024-09-11T07:53:06.751 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-11T07:53:06.752 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.create_missing_column_families: 0 2024-09-11T07:53:06.752 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.db_log_dir: 2024-09-11T07:53:06.752 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-b/store.db 2024-09-11T07:53:06.752 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-11T07:53:06.752 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-11T07:53:06.753 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-11T07:53:06.753 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-11T07:53:06.753 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-11T07:53:06.753 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-11T07:53:06.754 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.advise_random_on_open: 1 2024-09-11T07:53:06.754 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.db_write_buffer_size: 0 2024-09-11T07:53:06.754 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.write_buffer_manager: 0x559af8968780 2024-09-11T07:53:06.754 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-11T07:53:06.754 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-09-11T07:53:06.754 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-11T07:53:06.754 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-11T07:53:06.754 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.rate_limiter: (nil) 2024-09-11T07:53:06.754 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-11T07:53:06.754 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.wal_recovery_mode: 2 2024-09-11T07:53:06.754 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.enable_thread_tracking: 0 2024-09-11T07:53:06.754 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.enable_pipelined_write: 0 2024-09-11T07:53:06.754 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.unordered_write: 0 2024-09-11T07:53:06.754 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-11T07:53:06.755 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-11T07:53:06.755 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-11T07:53:06.755 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-11T07:53:06.755 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.row_cache: None 2024-09-11T07:53:06.755 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.wal_filter: None 2024-09-11T07:53:06.755 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-11T07:53:06.755 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.allow_ingest_behind: 0 2024-09-11T07:53:06.755 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.preserve_deletes: 0 2024-09-11T07:53:06.755 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.two_write_queues: 0 2024-09-11T07:53:06.755 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.manual_wal_flush: 0 2024-09-11T07:53:06.755 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.atomic_flush: 0 2024-09-11T07:53:06.755 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-11T07:53:06.755 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-11T07:53:06.755 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-11T07:53:06.756 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.log_readahead_size: 0 2024-09-11T07:53:06.756 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-11T07:53:06.756 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.best_efforts_recovery: 0 2024-09-11T07:53:06.756 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-11T07:53:06.756 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-11T07:53:06.756 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.allow_data_in_errors: 0 2024-09-11T07:53:06.756 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.db_host_id: __hostname__ 2024-09-11T07:53:06.756 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.max_background_jobs: 2 2024-09-11T07:53:06.756 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.max_background_compactions: -1 2024-09-11T07:53:06.756 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.max_subcompactions: 1 2024-09-11T07:53:06.756 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-11T07:53:06.756 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-11T07:53:06.756 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-11T07:53:06.756 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.max_total_wal_size: 0 2024-09-11T07:53:06.756 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-11T07:53:06.757 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-11T07:53:06.757 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-11T07:53:06.757 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-11T07:53:06.757 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.max_open_files: -1 2024-09-11T07:53:06.757 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.bytes_per_sync: 0 2024-09-11T07:53:06.757 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-11T07:53:06.757 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-11T07:53:06.757 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.compaction_readahead_size: 0 2024-09-11T07:53:06.757 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.max_background_flushes: -1 2024-09-11T07:53:06.757 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Compression algorithms supported: 2024-09-11T07:53:06.757 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: kZSTD supported: 0 2024-09-11T07:53:06.757 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: kXpressCompression supported: 0 2024-09-11T07:53:06.757 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: kBZip2Compression supported: 0 2024-09-11T07:53:06.758 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-11T07:53:06.758 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: kLZ4Compression supported: 1 2024-09-11T07:53:06.758 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: kZlibCompression supported: 1 2024-09-11T07:53:06.758 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: kLZ4HCCompression supported: 1 2024-09-11T07:53:06.758 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: kSnappyCompression supported: 1 2024-09-11T07:53:06.758 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-11T07:53:06.758 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000003 2024-09-11T07:53:06.758 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-09-11T07:53:06.758 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-11T07:53:06.758 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.merge_operator: 2024-09-11T07:53:06.758 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.compaction_filter: None 2024-09-11T07:53:06.758 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.compaction_filter_factory: None 2024-09-11T07:53:06.758 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.sst_partitioner_factory: None 2024-09-11T07:53:06.758 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-11T07:53:06.759 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-11T07:53:06.759 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x559af8851a38) 2024-09-11T07:53:06.759 INFO:journalctl@ceph.mon.b.smithi139.stdout: cache_index_and_filter_blocks: 1 2024-09-11T07:53:06.759 INFO:journalctl@ceph.mon.b.smithi139.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-11T07:53:06.759 INFO:journalctl@ceph.mon.b.smithi139.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-11T07:53:06.759 INFO:journalctl@ceph.mon.b.smithi139.stdout: pin_top_level_index_and_filter: 1 2024-09-11T07:53:06.759 INFO:journalctl@ceph.mon.b.smithi139.stdout: index_type: 0 2024-09-11T07:53:06.759 INFO:journalctl@ceph.mon.b.smithi139.stdout: data_block_index_type: 0 2024-09-11T07:53:06.759 INFO:journalctl@ceph.mon.b.smithi139.stdout: index_shortening: 1 2024-09-11T07:53:06.759 INFO:journalctl@ceph.mon.b.smithi139.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-11T07:53:06.759 INFO:journalctl@ceph.mon.b.smithi139.stdout: hash_index_allow_collision: 1 2024-09-11T07:53:06.759 INFO:journalctl@ceph.mon.b.smithi139.stdout: checksum: 1 2024-09-11T07:53:06.760 INFO:journalctl@ceph.mon.b.smithi139.stdout: no_block_cache: 0 2024-09-11T07:53:06.760 INFO:journalctl@ceph.mon.b.smithi139.stdout: block_cache: 0x559af8887350 2024-09-11T07:53:06.760 INFO:journalctl@ceph.mon.b.smithi139.stdout: block_cache_name: BinnedLRUCache 2024-09-11T07:53:06.760 INFO:journalctl@ceph.mon.b.smithi139.stdout: block_cache_options: 2024-09-11T07:53:06.760 INFO:journalctl@ceph.mon.b.smithi139.stdout: capacity : 536870912 2024-09-11T07:53:06.760 INFO:journalctl@ceph.mon.b.smithi139.stdout: num_shard_bits : 4 2024-09-11T07:53:06.760 INFO:journalctl@ceph.mon.b.smithi139.stdout: strict_capacity_limit : 0 2024-09-11T07:53:06.760 INFO:journalctl@ceph.mon.b.smithi139.stdout: high_pri_pool_ratio: 0.000 2024-09-11T07:53:06.760 INFO:journalctl@ceph.mon.b.smithi139.stdout: block_cache_compressed: (nil) 2024-09-11T07:53:06.760 INFO:journalctl@ceph.mon.b.smithi139.stdout: persistent_cache: (nil) 2024-09-11T07:53:06.760 INFO:journalctl@ceph.mon.b.smithi139.stdout: block_size: 4096 2024-09-11T07:53:06.760 INFO:journalctl@ceph.mon.b.smithi139.stdout: block_size_deviation: 10 2024-09-11T07:53:06.760 INFO:journalctl@ceph.mon.b.smithi139.stdout: block_restart_interval: 16 2024-09-11T07:53:06.760 INFO:journalctl@ceph.mon.b.smithi139.stdout: index_block_restart_interval: 1 2024-09-11T07:53:06.761 INFO:journalctl@ceph.mon.b.smithi139.stdout: metadata_block_size: 4096 2024-09-11T07:53:06.761 INFO:journalctl@ceph.mon.b.smithi139.stdout: partition_filters: 0 2024-09-11T07:53:06.761 INFO:journalctl@ceph.mon.b.smithi139.stdout: use_delta_encoding: 1 2024-09-11T07:53:06.761 INFO:journalctl@ceph.mon.b.smithi139.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-09-11T07:53:06.761 INFO:journalctl@ceph.mon.b.smithi139.stdout: whole_key_filtering: 1 2024-09-11T07:53:06.761 INFO:journalctl@ceph.mon.b.smithi139.stdout: verify_compression: 0 2024-09-11T07:53:06.761 INFO:journalctl@ceph.mon.b.smithi139.stdout: read_amp_bytes_per_bit: 0 2024-09-11T07:53:06.761 INFO:journalctl@ceph.mon.b.smithi139.stdout: format_version: 4 2024-09-11T07:53:06.761 INFO:journalctl@ceph.mon.b.smithi139.stdout: enable_index_compression: 1 2024-09-11T07:53:06.761 INFO:journalctl@ceph.mon.b.smithi139.stdout: block_align: 0 2024-09-11T07:53:06.761 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.write_buffer_size: 33554432 2024-09-11T07:53:06.761 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.max_write_buffer_number: 2 2024-09-11T07:53:06.761 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.compression: NoCompression 2024-09-11T07:53:06.761 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.bottommost_compression: Disabled 2024-09-11T07:53:06.761 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.prefix_extractor: nullptr 2024-09-11T07:53:06.762 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-11T07:53:06.762 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.num_levels: 7 2024-09-11T07:53:06.762 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-11T07:53:06.762 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-11T07:53:06.762 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-11T07:53:06.762 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-11T07:53:06.762 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-11T07:53:06.762 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-11T07:53:06.762 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-11T07:53:06.762 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-11T07:53:06.762 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-11T07:53:06.762 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-11T07:53:06.762 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-11T07:53:06.762 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.compression_opts.level: 32767 2024-09-11T07:53:06.763 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.compression_opts.strategy: 0 2024-09-11T07:53:06.763 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-11T07:53:06.763 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-11T07:53:06.763 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-11T07:53:06.763 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.compression_opts.enabled: false 2024-09-11T07:53:06.763 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-11T07:53:06.763 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-11T07:53:06.763 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-11T07:53:06.763 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.target_file_size_base: 67108864 2024-09-11T07:53:06.763 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-11T07:53:06.763 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-11T07:53:06.763 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-11T07:53:06.763 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-11T07:53:06.763 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-11T07:53:06.764 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-11T07:53:06.764 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-11T07:53:06.764 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-11T07:53:06.764 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-11T07:53:06.764 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-11T07:53:06.764 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-11T07:53:06.764 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-11T07:53:06.764 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-11T07:53:06.764 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.arena_block_size: 4194304 2024-09-11T07:53:06.764 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-11T07:53:06.764 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-11T07:53:06.764 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-09-11T07:53:06.764 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.disable_auto_compactions: 0 2024-09-11T07:53:06.764 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-11T07:53:06.765 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-11T07:53:06.765 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-11T07:53:06.765 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-11T07:53:06.765 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-11T07:53:06.765 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-11T07:53:06.765 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-11T07:53:06.765 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-11T07:53:06.765 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-11T07:53:06.765 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-11T07:53:06.765 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-11T07:53:06.765 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.inplace_update_support: 0 2024-09-11T07:53:06.765 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-11T07:53:06.765 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-11T07:53:06.765 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-11T07:53:06.766 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-11T07:53:06.766 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.bloom_locality: 0 2024-09-11T07:53:06.766 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.max_successive_merges: 0 2024-09-11T07:53:06.766 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-11T07:53:06.766 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.paranoid_file_checks: 0 2024-09-11T07:53:06.766 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.force_consistency_checks: 1 2024-09-11T07:53:06.766 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.report_bg_io_stats: 0 2024-09-11T07:53:06.766 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.ttl: 2592000 2024-09-11T07:53:06.766 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-11T07:53:06.766 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.enable_blob_files: false 2024-09-11T07:53:06.766 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.min_blob_size: 0 2024-09-11T07:53:06.766 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.blob_file_size: 268435456 2024-09-11T07:53:06.766 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-11T07:53:06.766 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-11T07:53:06.766 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-11T07:53:06.767 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000003 succeeded,manifest_file_number is 3, next_file_number is 5, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-11T07:53:06.767 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 0 2024-09-11T07:53:06.767 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: [db/version_set.cc:4082] Creating manifest 7 2024-09-11T07:53:06.767 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726041186459053, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-11T07:53:06.767 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #4 mode 2 2024-09-11T07:53:06.767 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-09-11T07:53:06.767 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726041186459452, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1571, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1726041186, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "44314c10-d41a-4eac-ae68-6e57c06464d3", "db_session_id": "0YLXSRV0J7B4MNH02AGW"}} 2024-09-11T07:53:06.767 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: [db/version_set.cc:4082] Creating manifest 9 2024-09-11T07:53:06.767 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726041186459786, "job": 1, "event": "recovery_finished"} 2024-09-11T07:53:06.767 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-11T07:53:06.767 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x559af88ba700 2024-09-11T07:53:06.767 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: DB pointer 0x559af888a000 2024-09-11T07:53:06.767 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-09-11T07:53:06.767 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-09-11T07:53:06.768 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-09-11T07:53:06.768 INFO:journalctl@ceph.mon.b.smithi139.stdout: ** DB Stats ** 2024-09-11T07:53:06.768 INFO:journalctl@ceph.mon.b.smithi139.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T07:53:06.768 INFO:journalctl@ceph.mon.b.smithi139.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-11T07:53:06.768 INFO:journalctl@ceph.mon.b.smithi139.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-11T07:53:06.768 INFO:journalctl@ceph.mon.b.smithi139.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-11T07:53:06.768 INFO:journalctl@ceph.mon.b.smithi139.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-11T07:53:06.768 INFO:journalctl@ceph.mon.b.smithi139.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-09-11T07:53:06.768 INFO:journalctl@ceph.mon.b.smithi139.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-11T07:53:06.768 INFO:journalctl@ceph.mon.b.smithi139.stdout: 2024-09-11T07:53:06.768 INFO:journalctl@ceph.mon.b.smithi139.stdout: ** Compaction Stats [default] ** 2024-09-11T07:53:06.768 INFO:journalctl@ceph.mon.b.smithi139.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T07:53:06.768 INFO:journalctl@ceph.mon.b.smithi139.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T07:53:06.768 INFO:journalctl@ceph.mon.b.smithi139.stdout: L0 1/0 1.53 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 2024-09-11T07:53:06.769 INFO:journalctl@ceph.mon.b.smithi139.stdout: Sum 1/0 1.53 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 2024-09-11T07:53:06.769 INFO:journalctl@ceph.mon.b.smithi139.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 2024-09-11T07:53:06.769 INFO:journalctl@ceph.mon.b.smithi139.stdout: 2024-09-11T07:53:06.769 INFO:journalctl@ceph.mon.b.smithi139.stdout: ** Compaction Stats [default] ** 2024-09-11T07:53:06.769 INFO:journalctl@ceph.mon.b.smithi139.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T07:53:06.769 INFO:journalctl@ceph.mon.b.smithi139.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T07:53:06.769 INFO:journalctl@ceph.mon.b.smithi139.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.9 0.00 0.00 1 0.000 0 0 2024-09-11T07:53:06.769 INFO:journalctl@ceph.mon.b.smithi139.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T07:53:06.769 INFO:journalctl@ceph.mon.b.smithi139.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-11T07:53:06.769 INFO:journalctl@ceph.mon.b.smithi139.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-11T07:53:06.769 INFO:journalctl@ceph.mon.b.smithi139.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-11T07:53:06.769 INFO:journalctl@ceph.mon.b.smithi139.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-11T07:53:06.769 INFO:journalctl@ceph.mon.b.smithi139.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-11T07:53:06.770 INFO:journalctl@ceph.mon.b.smithi139.stdout: Cumulative compaction: 0.00 GB write, 0.55 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T07:53:06.770 INFO:journalctl@ceph.mon.b.smithi139.stdout: Interval compaction: 0.00 GB write, 0.55 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T07:53:06.770 INFO:journalctl@ceph.mon.b.smithi139.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-11T07:53:06.770 INFO:journalctl@ceph.mon.b.smithi139.stdout: 2024-09-11T07:53:06.770 INFO:journalctl@ceph.mon.b.smithi139.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-11T07:53:06.770 INFO:journalctl@ceph.mon.b.smithi139.stdout: 2024-09-11T07:53:06.770 INFO:journalctl@ceph.mon.b.smithi139.stdout: ** Compaction Stats [default] ** 2024-09-11T07:53:06.770 INFO:journalctl@ceph.mon.b.smithi139.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T07:53:06.770 INFO:journalctl@ceph.mon.b.smithi139.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T07:53:06.770 INFO:journalctl@ceph.mon.b.smithi139.stdout: L0 1/0 1.53 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 2024-09-11T07:53:06.770 INFO:journalctl@ceph.mon.b.smithi139.stdout: Sum 1/0 1.53 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 2024-09-11T07:53:06.770 INFO:journalctl@ceph.mon.b.smithi139.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-09-11T07:53:06.770 INFO:journalctl@ceph.mon.b.smithi139.stdout: 2024-09-11T07:53:06.770 INFO:journalctl@ceph.mon.b.smithi139.stdout: ** Compaction Stats [default] ** 2024-09-11T07:53:06.771 INFO:journalctl@ceph.mon.b.smithi139.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T07:53:06.771 INFO:journalctl@ceph.mon.b.smithi139.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T07:53:06.771 INFO:journalctl@ceph.mon.b.smithi139.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.9 0.00 0.00 1 0.000 0 0 2024-09-11T07:53:06.771 INFO:journalctl@ceph.mon.b.smithi139.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T07:53:06.771 INFO:journalctl@ceph.mon.b.smithi139.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-11T07:53:06.771 INFO:journalctl@ceph.mon.b.smithi139.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-11T07:53:06.771 INFO:journalctl@ceph.mon.b.smithi139.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-11T07:53:06.771 INFO:journalctl@ceph.mon.b.smithi139.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-11T07:53:06.771 INFO:journalctl@ceph.mon.b.smithi139.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-11T07:53:06.771 INFO:journalctl@ceph.mon.b.smithi139.stdout: Cumulative compaction: 0.00 GB write, 0.54 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T07:53:06.771 INFO:journalctl@ceph.mon.b.smithi139.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T07:53:06.771 INFO:journalctl@ceph.mon.b.smithi139.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-11T07:53:06.771 INFO:journalctl@ceph.mon.b.smithi139.stdout: 2024-09-11T07:53:06.771 INFO:journalctl@ceph.mon.b.smithi139.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-11T07:53:06.772 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: using public_addr v2:172.21.15.139:0/0 -> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] 2024-09-11T07:53:06.772 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: starting mon.b rank -1 at public addrs [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] at bind addrs [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 41bd1096-7012-11ef-bcea-c7b262605968 2024-09-11T07:53:06.772 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: mon.b@-1(???) e0 preinit fsid 41bd1096-7012-11ef-bcea-c7b262605968 2024-09-11T07:53:06.772 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: mon.b@-1(synchronizing).mds e1 new map 2024-09-11T07:53:06.772 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-11T07:53:06.772 INFO:journalctl@ceph.mon.b.smithi139.stdout: e1 2024-09-11T07:53:06.772 INFO:journalctl@ceph.mon.b.smithi139.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-11T07:53:06.772 INFO:journalctl@ceph.mon.b.smithi139.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-11T07:53:06.772 INFO:journalctl@ceph.mon.b.smithi139.stdout: legacy client fscid: -1 2024-09-11T07:53:06.772 INFO:journalctl@ceph.mon.b.smithi139.stdout: 2024-09-11T07:53:06.772 INFO:journalctl@ceph.mon.b.smithi139.stdout: No filesystems configured 2024-09-11T07:53:06.772 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-11T07:53:06.772 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-11T07:53:06.772 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-11T07:53:06.773 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-11T07:53:06.773 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-11T07:53:06.773 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-11T07:53:06.773 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-11T07:53:06.773 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T07:53:06.773 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T07:53:06.773 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T07:53:06.773 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:06.773 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: from='client.? 172.21.15.110:0/2982156807' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-11T07:53:06.773 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T07:53:06.773 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:06.773 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:06.773 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:06.773 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:06.774 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:53:06.774 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:53:06.774 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:53:06.774 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:06.774 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T07:53:06.774 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:53:06.774 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi110:172.21.15.110=a;smithi110:[v2:172.21.15.110:3301,v1:172.21.15.110:6790]=c;smithi139:172.21.15.139=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:53:06.774 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: Saving service mon spec with placement smithi110:172.21.15.110=a;smithi110:[v2:172.21.15.110:3301,v1:172.21.15.110:6790]=c;smithi139:172.21.15.139=b;count:3 2024-09-11T07:53:06.774 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: Deploying daemon mon.b on smithi139 2024-09-11T07:53:06.774 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:06.774 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:06.774 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:06.774 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:06.775 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:06.775 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 ceph-mon[27311]: mon.b@-1(synchronizing).paxosservice(auth 1..5) refresh upgraded, format 0 -> 3 2024-09-11T07:53:06.775 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 bash[27275]: 4372b142090407d319454577968ec4111fd04b600ecedf8fd0bdf2cc2aec6f80 2024-09-11T07:53:06.775 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:06 smithi139 systemd[1]: Started Ceph mon.b for 41bd1096-7012-11ef-bcea-c7b262605968. 2024-09-11T07:53:07.321 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T07:53:07.321 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T07:53:07.588 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:53:11.922 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:11 smithi110 ceph-mon[25392]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:11.922 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:11 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:53:11.922 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:11 smithi110 ceph-mon[25392]: mon.a calling monitor election 2024-09-11T07:53:11.922 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:11 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:53:11.922 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:11 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:53:11.922 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:11 smithi110 ceph-mon[25392]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:11.922 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:11 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:53:11.922 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:11 smithi110 ceph-mon[25392]: mon.b calling monitor election 2024-09-11T07:53:11.922 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:11 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:53:11.922 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:11 smithi110 ceph-mon[25392]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:11.922 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:11 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:53:11.923 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:11 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:53:11.923 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:11 smithi110 ceph-mon[25392]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-11T07:53:11.923 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:11 smithi110 ceph-mon[25392]: monmap e2: 2 mons at {a=[v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0],b=[v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-11T07:53:11.923 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:11 smithi110 ceph-mon[25392]: fsmap 2024-09-11T07:53:11.923 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:11 smithi110 ceph-mon[25392]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T07:53:11.923 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:11 smithi110 ceph-mon[25392]: mgrmap e13: y(active, since 39s) 2024-09-11T07:53:11.923 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:11 smithi110 ceph-mon[25392]: overall HEALTH_OK 2024-09-11T07:53:11.923 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:11 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:11.923 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:11 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:11.923 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:11 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:11.923 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:11 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T07:53:11.923 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:11 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:53:11.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:11 smithi139 ceph-mon[27311]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:11.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:11 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:53:11.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:11 smithi139 ceph-mon[27311]: mon.a calling monitor election 2024-09-11T07:53:11.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:11 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:53:11.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:11 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:53:12.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:11 smithi139 ceph-mon[27311]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:12.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:11 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:53:12.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:11 smithi139 ceph-mon[27311]: mon.b calling monitor election 2024-09-11T07:53:12.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:11 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:53:12.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:11 smithi139 ceph-mon[27311]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:12.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:11 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:53:12.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:11 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:53:12.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:11 smithi139 ceph-mon[27311]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-11T07:53:12.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:11 smithi139 ceph-mon[27311]: monmap e2: 2 mons at {a=[v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0],b=[v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-11T07:53:12.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:11 smithi139 ceph-mon[27311]: fsmap 2024-09-11T07:53:12.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:11 smithi139 ceph-mon[27311]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T07:53:12.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:11 smithi139 ceph-mon[27311]: mgrmap e13: y(active, since 39s) 2024-09-11T07:53:12.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:11 smithi139 ceph-mon[27311]: overall HEALTH_OK 2024-09-11T07:53:12.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:11 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:12.001 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:11 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:12.001 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:11 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:12.001 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:11 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T07:53:12.001 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:11 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:53:12.212 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-11T07:53:12.212 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":2,"fsid":"41bd1096-7012-11ef-bcea-c7b262605968","modified":"2024-09-11T07:53:06.484202Z","created":"2024-09-11T07:51:39.629440Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:3300","nonce":0},{"type":"v1","addr":"172.21.15.110:6789","nonce":0}]},"addr":"172.21.15.110:6789/0","public_addr":"172.21.15.110:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:3300","nonce":0},{"type":"v1","addr":"172.21.15.139:6789","nonce":0}]},"addr":"172.21.15.139:6789/0","public_addr":"172.21.15.139:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-11T07:53:12.212 INFO:teuthology.orchestra.run.smithi139.stderr:dumped monmap epoch 2 2024-09-11T07:53:12.885 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:12 smithi110 ceph-mon[25392]: Deploying daemon mon.c on smithi110 2024-09-11T07:53:12.885 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:12 smithi110 ceph-mon[25392]: from='client.? 172.21.15.139:0/1309787194' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T07:53:12.885 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:12 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:53:12.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:12 smithi139 ceph-mon[27311]: Deploying daemon mon.c on smithi110 2024-09-11T07:53:12.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:12 smithi139 ceph-mon[27311]: from='client.? 172.21.15.139:0/1309787194' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T07:53:12.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:12 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:53:13.924 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:13 smithi110 ceph-mon[25392]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:13.998 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:13 smithi139 ceph-mon[27311]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:14.050 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T07:53:14.050 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T07:53:14.298 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:53:14.468 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 systemd[1]: Starting Ceph mon.c for 41bd1096-7012-11ef-bcea-c7b262605968... 2024-09-11T07:53:14.757 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 podman[31689]: 2024-09-11 07:53:14.408419753 +0000 UTC m=+0.015399379 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T07:53:14.757 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 podman[31689]: 2024-09-11 07:53:14.509253779 +0000 UTC m=+0.116233432 container create 6c2b0e04ed670ee58c9b649addec5719a6067843abb9f26435d8f00388bbb113 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-mon-c, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-11T07:53:14.758 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 podman[31689]: 2024-09-11 07:53:14.728546154 +0000 UTC m=+0.335525762 container init 6c2b0e04ed670ee58c9b649addec5719a6067843abb9f26435d8f00388bbb113 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-mon-c, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T07:53:14.758 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 podman[31689]: 2024-09-11 07:53:14.731925281 +0000 UTC m=+0.338904900 container start 6c2b0e04ed670ee58c9b649addec5719a6067843abb9f26435d8f00388bbb113 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-mon-c, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T07:53:14.758 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: set uid:gid to 167:167 (ceph:ceph) 2024-09-11T07:53:14.758 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable), process ceph-mon, pid 2 2024-09-11T07:53:14.758 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: pidfile_write: ignore empty --pid-file 2024-09-11T07:53:14.758 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: load: jerasure load: lrc 2024-09-11T07:53:14.758 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: RocksDB version: 6.15.5 2024-09-11T07:53:14.758 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-09-11T07:53:14.758 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Compile date Sep 10 2024 2024-09-11T07:53:14.758 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: DB SUMMARY 2024-09-11T07:53:14.759 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: DB Session ID: HLIY92S6KWC52E3DRIEQ 2024-09-11T07:53:14.759 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: CURRENT file: CURRENT 2024-09-11T07:53:14.759 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: IDENTITY file: IDENTITY 2024-09-11T07:53:14.759 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: MANIFEST file: MANIFEST-000003 size: 57 Bytes 2024-09-11T07:53:14.759 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-11T07:53:14.759 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 636 ; 2024-09-11T07:53:14.759 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.error_if_exists: 0 2024-09-11T07:53:14.759 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.create_if_missing: 0 2024-09-11T07:53:14.759 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.paranoid_checks: 1 2024-09-11T07:53:14.759 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-11T07:53:14.759 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.env: 0x563ac1098440 2024-09-11T07:53:14.759 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.fs: Posix File System 2024-09-11T07:53:14.760 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.info_log: 0x563ac2908ec0 2024-09-11T07:53:14.760 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.max_file_opening_threads: 16 2024-09-11T07:53:14.760 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.statistics: (nil) 2024-09-11T07:53:14.760 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.use_fsync: 0 2024-09-11T07:53:14.760 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.max_log_file_size: 0 2024-09-11T07:53:14.760 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-11T07:53:14.760 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-11T07:53:14.760 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.keep_log_file_num: 1000 2024-09-11T07:53:14.760 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.recycle_log_file_num: 0 2024-09-11T07:53:14.760 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.allow_fallocate: 1 2024-09-11T07:53:14.760 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.allow_mmap_reads: 0 2024-09-11T07:53:14.760 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.allow_mmap_writes: 0 2024-09-11T07:53:14.761 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.use_direct_reads: 0 2024-09-11T07:53:14.761 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-11T07:53:14.761 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.create_missing_column_families: 0 2024-09-11T07:53:14.761 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.db_log_dir: 2024-09-11T07:53:14.761 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-c/store.db 2024-09-11T07:53:14.761 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-11T07:53:14.761 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-11T07:53:14.761 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-11T07:53:14.761 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-11T07:53:14.761 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-11T07:53:14.761 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-11T07:53:14.761 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.advise_random_on_open: 1 2024-09-11T07:53:14.761 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.db_write_buffer_size: 0 2024-09-11T07:53:14.762 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.write_buffer_manager: 0x563ac29c67b0 2024-09-11T07:53:14.762 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-11T07:53:14.762 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-09-11T07:53:14.762 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-11T07:53:14.762 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-11T07:53:14.762 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.rate_limiter: (nil) 2024-09-11T07:53:14.763 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-11T07:53:14.763 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.wal_recovery_mode: 2 2024-09-11T07:53:14.763 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.enable_thread_tracking: 0 2024-09-11T07:53:14.763 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.enable_pipelined_write: 0 2024-09-11T07:53:14.763 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.unordered_write: 0 2024-09-11T07:53:14.763 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-11T07:53:14.763 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-11T07:53:14.764 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-11T07:53:14.764 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-11T07:53:14.764 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.row_cache: None 2024-09-11T07:53:14.764 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.wal_filter: None 2024-09-11T07:53:14.764 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-11T07:53:14.764 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.allow_ingest_behind: 0 2024-09-11T07:53:14.764 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.preserve_deletes: 0 2024-09-11T07:53:14.764 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.two_write_queues: 0 2024-09-11T07:53:14.764 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.manual_wal_flush: 0 2024-09-11T07:53:14.764 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.atomic_flush: 0 2024-09-11T07:53:14.764 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-11T07:53:14.764 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-11T07:53:14.764 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-11T07:53:14.764 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.log_readahead_size: 0 2024-09-11T07:53:14.765 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-11T07:53:14.765 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.best_efforts_recovery: 0 2024-09-11T07:53:14.765 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-11T07:53:14.765 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-11T07:53:14.765 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.allow_data_in_errors: 0 2024-09-11T07:53:14.765 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.db_host_id: __hostname__ 2024-09-11T07:53:14.765 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.max_background_jobs: 2 2024-09-11T07:53:14.765 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.max_background_compactions: -1 2024-09-11T07:53:14.765 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.max_subcompactions: 1 2024-09-11T07:53:14.765 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-11T07:53:14.765 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-11T07:53:14.765 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-11T07:53:14.765 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.max_total_wal_size: 0 2024-09-11T07:53:14.765 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-11T07:53:14.765 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-11T07:53:14.766 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-11T07:53:14.766 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-11T07:53:14.766 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.max_open_files: -1 2024-09-11T07:53:14.766 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.bytes_per_sync: 0 2024-09-11T07:53:14.766 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-11T07:53:14.766 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-11T07:53:14.766 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.compaction_readahead_size: 0 2024-09-11T07:53:14.766 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.max_background_flushes: -1 2024-09-11T07:53:14.766 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Compression algorithms supported: 2024-09-11T07:53:14.766 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: kZSTD supported: 0 2024-09-11T07:53:14.766 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: kXpressCompression supported: 0 2024-09-11T07:53:14.766 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: kBZip2Compression supported: 0 2024-09-11T07:53:14.766 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-11T07:53:14.766 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: kLZ4Compression supported: 1 2024-09-11T07:53:14.766 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: kZlibCompression supported: 1 2024-09-11T07:53:14.766 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: kLZ4HCCompression supported: 1 2024-09-11T07:53:14.767 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: kSnappyCompression supported: 1 2024-09-11T07:53:14.767 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-11T07:53:14.767 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000003 2024-09-11T07:53:14.767 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-09-11T07:53:14.767 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-11T07:53:14.767 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.merge_operator: 2024-09-11T07:53:14.767 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.compaction_filter: None 2024-09-11T07:53:14.767 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.compaction_filter_factory: None 2024-09-11T07:53:14.767 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.sst_partitioner_factory: None 2024-09-11T07:53:14.767 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-11T07:53:14.767 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-11T07:53:14.767 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x563ac28afa38) 2024-09-11T07:53:14.767 INFO:journalctl@ceph.mon.c.smithi110.stdout: cache_index_and_filter_blocks: 1 2024-09-11T07:53:14.768 INFO:journalctl@ceph.mon.c.smithi110.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-11T07:53:14.768 INFO:journalctl@ceph.mon.c.smithi110.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-11T07:53:14.768 INFO:journalctl@ceph.mon.c.smithi110.stdout: pin_top_level_index_and_filter: 1 2024-09-11T07:53:14.768 INFO:journalctl@ceph.mon.c.smithi110.stdout: index_type: 0 2024-09-11T07:53:14.768 INFO:journalctl@ceph.mon.c.smithi110.stdout: data_block_index_type: 0 2024-09-11T07:53:14.768 INFO:journalctl@ceph.mon.c.smithi110.stdout: index_shortening: 1 2024-09-11T07:53:14.768 INFO:journalctl@ceph.mon.c.smithi110.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-11T07:53:14.768 INFO:journalctl@ceph.mon.c.smithi110.stdout: hash_index_allow_collision: 1 2024-09-11T07:53:14.768 INFO:journalctl@ceph.mon.c.smithi110.stdout: checksum: 1 2024-09-11T07:53:14.768 INFO:journalctl@ceph.mon.c.smithi110.stdout: no_block_cache: 0 2024-09-11T07:53:14.768 INFO:journalctl@ceph.mon.c.smithi110.stdout: block_cache: 0x563ac28e51f0 2024-09-11T07:53:14.768 INFO:journalctl@ceph.mon.c.smithi110.stdout: block_cache_name: BinnedLRUCache 2024-09-11T07:53:14.768 INFO:journalctl@ceph.mon.c.smithi110.stdout: block_cache_options: 2024-09-11T07:53:14.768 INFO:journalctl@ceph.mon.c.smithi110.stdout: capacity : 536870912 2024-09-11T07:53:14.769 INFO:journalctl@ceph.mon.c.smithi110.stdout: num_shard_bits : 4 2024-09-11T07:53:14.769 INFO:journalctl@ceph.mon.c.smithi110.stdout: strict_capacity_limit : 0 2024-09-11T07:53:14.769 INFO:journalctl@ceph.mon.c.smithi110.stdout: high_pri_pool_ratio: 0.000 2024-09-11T07:53:14.769 INFO:journalctl@ceph.mon.c.smithi110.stdout: block_cache_compressed: (nil) 2024-09-11T07:53:14.769 INFO:journalctl@ceph.mon.c.smithi110.stdout: persistent_cache: (nil) 2024-09-11T07:53:14.769 INFO:journalctl@ceph.mon.c.smithi110.stdout: block_size: 4096 2024-09-11T07:53:14.769 INFO:journalctl@ceph.mon.c.smithi110.stdout: block_size_deviation: 10 2024-09-11T07:53:14.769 INFO:journalctl@ceph.mon.c.smithi110.stdout: block_restart_interval: 16 2024-09-11T07:53:14.769 INFO:journalctl@ceph.mon.c.smithi110.stdout: index_block_restart_interval: 1 2024-09-11T07:53:14.769 INFO:journalctl@ceph.mon.c.smithi110.stdout: metadata_block_size: 4096 2024-09-11T07:53:14.769 INFO:journalctl@ceph.mon.c.smithi110.stdout: partition_filters: 0 2024-09-11T07:53:14.769 INFO:journalctl@ceph.mon.c.smithi110.stdout: use_delta_encoding: 1 2024-09-11T07:53:14.770 INFO:journalctl@ceph.mon.c.smithi110.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-09-11T07:53:14.770 INFO:journalctl@ceph.mon.c.smithi110.stdout: whole_key_filtering: 1 2024-09-11T07:53:14.770 INFO:journalctl@ceph.mon.c.smithi110.stdout: verify_compression: 0 2024-09-11T07:53:14.770 INFO:journalctl@ceph.mon.c.smithi110.stdout: read_amp_bytes_per_bit: 0 2024-09-11T07:53:14.770 INFO:journalctl@ceph.mon.c.smithi110.stdout: format_version: 4 2024-09-11T07:53:14.770 INFO:journalctl@ceph.mon.c.smithi110.stdout: enable_index_compression: 1 2024-09-11T07:53:14.770 INFO:journalctl@ceph.mon.c.smithi110.stdout: block_align: 0 2024-09-11T07:53:14.770 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.write_buffer_size: 33554432 2024-09-11T07:53:14.770 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.max_write_buffer_number: 2 2024-09-11T07:53:14.770 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.compression: NoCompression 2024-09-11T07:53:14.770 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.bottommost_compression: Disabled 2024-09-11T07:53:14.770 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.prefix_extractor: nullptr 2024-09-11T07:53:14.770 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-11T07:53:14.770 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.num_levels: 7 2024-09-11T07:53:14.771 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-11T07:53:14.771 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-11T07:53:14.771 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-11T07:53:14.771 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-11T07:53:14.771 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-11T07:53:14.771 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-11T07:53:14.771 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-11T07:53:14.771 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-11T07:53:14.771 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-11T07:53:14.771 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-11T07:53:14.771 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-11T07:53:14.771 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.compression_opts.level: 32767 2024-09-11T07:53:14.771 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.compression_opts.strategy: 0 2024-09-11T07:53:14.772 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-11T07:53:14.772 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-11T07:53:14.772 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-11T07:53:14.772 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.compression_opts.enabled: false 2024-09-11T07:53:14.772 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-11T07:53:14.772 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-11T07:53:14.772 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-11T07:53:14.772 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.target_file_size_base: 67108864 2024-09-11T07:53:14.772 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-11T07:53:14.772 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-11T07:53:14.772 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-11T07:53:14.772 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-11T07:53:14.772 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-11T07:53:14.773 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-11T07:53:14.773 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-11T07:53:14.773 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-11T07:53:14.773 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-11T07:53:14.773 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-11T07:53:14.773 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-11T07:53:14.773 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-11T07:53:14.773 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-11T07:53:14.773 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.arena_block_size: 4194304 2024-09-11T07:53:14.773 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-11T07:53:14.773 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-11T07:53:14.773 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-09-11T07:53:14.773 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.disable_auto_compactions: 0 2024-09-11T07:53:14.774 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-11T07:53:14.774 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-11T07:53:14.774 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-11T07:53:14.774 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-11T07:53:14.774 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-11T07:53:14.774 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-11T07:53:14.774 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-11T07:53:14.774 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-11T07:53:14.774 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-11T07:53:14.774 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-11T07:53:14.774 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-11T07:53:14.774 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.inplace_update_support: 0 2024-09-11T07:53:14.774 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-11T07:53:14.775 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-11T07:53:14.775 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-11T07:53:14.775 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-11T07:53:14.775 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.bloom_locality: 0 2024-09-11T07:53:14.775 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.max_successive_merges: 0 2024-09-11T07:53:14.775 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-11T07:53:14.775 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.paranoid_file_checks: 0 2024-09-11T07:53:14.775 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.force_consistency_checks: 1 2024-09-11T07:53:14.775 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.report_bg_io_stats: 0 2024-09-11T07:53:14.775 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.ttl: 2592000 2024-09-11T07:53:14.775 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-11T07:53:14.775 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.enable_blob_files: false 2024-09-11T07:53:14.775 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.min_blob_size: 0 2024-09-11T07:53:14.775 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.blob_file_size: 268435456 2024-09-11T07:53:14.775 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-11T07:53:14.776 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-11T07:53:14.776 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-11T07:53:14.776 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000003 succeeded,manifest_file_number is 3, next_file_number is 5, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-11T07:53:14.776 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 0 2024-09-11T07:53:14.776 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: [db/version_set.cc:4082] Creating manifest 7 2024-09-11T07:53:14.776 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726041194755467, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-11T07:53:14.776 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #4 mode 2 2024-09-11T07:53:14.776 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-09-11T07:53:14.776 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726041194755884, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1696, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1726041194, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "af6049b5-11f6-48d8-a119-6423847c8c40", "db_session_id": "HLIY92S6KWC52E3DRIEQ"}} 2024-09-11T07:53:14.776 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: [db/version_set.cc:4082] Creating manifest 9 2024-09-11T07:53:14.776 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726041194756218, "job": 1, "event": "recovery_finished"} 2024-09-11T07:53:14.776 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-11T07:53:14.776 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x563ac2918700 2024-09-11T07:53:14.777 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: DB pointer 0x563ac28e8000 2024-09-11T07:53:14.777 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-09-11T07:53:14.777 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-09-11T07:53:14.777 INFO:journalctl@ceph.mon.c.smithi110.stdout: ** DB Stats ** 2024-09-11T07:53:14.777 INFO:journalctl@ceph.mon.c.smithi110.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T07:53:14.777 INFO:journalctl@ceph.mon.c.smithi110.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-11T07:53:14.777 INFO:journalctl@ceph.mon.c.smithi110.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-11T07:53:14.777 INFO:journalctl@ceph.mon.c.smithi110.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-11T07:53:14.777 INFO:journalctl@ceph.mon.c.smithi110.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-11T07:53:14.777 INFO:journalctl@ceph.mon.c.smithi110.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-09-11T07:53:14.777 INFO:journalctl@ceph.mon.c.smithi110.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-11T07:53:14.777 INFO:journalctl@ceph.mon.c.smithi110.stdout: 2024-09-11T07:53:14.777 INFO:journalctl@ceph.mon.c.smithi110.stdout: ** Compaction Stats [default] ** 2024-09-11T07:53:14.777 INFO:journalctl@ceph.mon.c.smithi110.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T07:53:14.777 INFO:journalctl@ceph.mon.c.smithi110.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T07:53:14.778 INFO:journalctl@ceph.mon.c.smithi110.stdout: L0 1/0 1.66 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 2024-09-11T07:53:14.778 INFO:journalctl@ceph.mon.c.smithi110.stdout: Sum 1/0 1.66 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 2024-09-11T07:53:14.778 INFO:journalctl@ceph.mon.c.smithi110.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 2024-09-11T07:53:14.778 INFO:journalctl@ceph.mon.c.smithi110.stdout: 2024-09-11T07:53:14.778 INFO:journalctl@ceph.mon.c.smithi110.stdout: ** Compaction Stats [default] ** 2024-09-11T07:53:14.778 INFO:journalctl@ceph.mon.c.smithi110.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T07:53:14.778 INFO:journalctl@ceph.mon.c.smithi110.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T07:53:14.778 INFO:journalctl@ceph.mon.c.smithi110.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.0 0.00 0.00 1 0.000 0 0 2024-09-11T07:53:14.778 INFO:journalctl@ceph.mon.c.smithi110.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T07:53:14.778 INFO:journalctl@ceph.mon.c.smithi110.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-11T07:53:14.778 INFO:journalctl@ceph.mon.c.smithi110.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-11T07:53:14.778 INFO:journalctl@ceph.mon.c.smithi110.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-11T07:53:14.778 INFO:journalctl@ceph.mon.c.smithi110.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-11T07:53:14.778 INFO:journalctl@ceph.mon.c.smithi110.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-11T07:53:14.779 INFO:journalctl@ceph.mon.c.smithi110.stdout: Cumulative compaction: 0.00 GB write, 0.59 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T07:53:14.779 INFO:journalctl@ceph.mon.c.smithi110.stdout: Interval compaction: 0.00 GB write, 0.59 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T07:53:14.779 INFO:journalctl@ceph.mon.c.smithi110.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-11T07:53:14.779 INFO:journalctl@ceph.mon.c.smithi110.stdout: 2024-09-11T07:53:14.779 INFO:journalctl@ceph.mon.c.smithi110.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-11T07:53:14.779 INFO:journalctl@ceph.mon.c.smithi110.stdout: 2024-09-11T07:53:14.779 INFO:journalctl@ceph.mon.c.smithi110.stdout: ** Compaction Stats [default] ** 2024-09-11T07:53:14.779 INFO:journalctl@ceph.mon.c.smithi110.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T07:53:14.779 INFO:journalctl@ceph.mon.c.smithi110.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T07:53:14.779 INFO:journalctl@ceph.mon.c.smithi110.stdout: L0 1/0 1.66 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 2024-09-11T07:53:14.779 INFO:journalctl@ceph.mon.c.smithi110.stdout: Sum 1/0 1.66 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 2024-09-11T07:53:14.779 INFO:journalctl@ceph.mon.c.smithi110.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-09-11T07:53:14.779 INFO:journalctl@ceph.mon.c.smithi110.stdout: 2024-09-11T07:53:14.779 INFO:journalctl@ceph.mon.c.smithi110.stdout: ** Compaction Stats [default] ** 2024-09-11T07:53:14.780 INFO:journalctl@ceph.mon.c.smithi110.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T07:53:14.780 INFO:journalctl@ceph.mon.c.smithi110.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T07:53:14.780 INFO:journalctl@ceph.mon.c.smithi110.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.0 0.00 0.00 1 0.000 0 0 2024-09-11T07:53:14.780 INFO:journalctl@ceph.mon.c.smithi110.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T07:53:14.780 INFO:journalctl@ceph.mon.c.smithi110.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-11T07:53:14.780 INFO:journalctl@ceph.mon.c.smithi110.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-11T07:53:14.780 INFO:journalctl@ceph.mon.c.smithi110.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-11T07:53:14.780 INFO:journalctl@ceph.mon.c.smithi110.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-11T07:53:14.780 INFO:journalctl@ceph.mon.c.smithi110.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-11T07:53:14.780 INFO:journalctl@ceph.mon.c.smithi110.stdout: Cumulative compaction: 0.00 GB write, 0.59 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T07:53:14.780 INFO:journalctl@ceph.mon.c.smithi110.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T07:53:14.780 INFO:journalctl@ceph.mon.c.smithi110.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-11T07:53:14.781 INFO:journalctl@ceph.mon.c.smithi110.stdout: 2024-09-11T07:53:14.781 INFO:journalctl@ceph.mon.c.smithi110.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-11T07:53:14.781 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-11T07:53:14.781 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: using public_addrv [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] 2024-09-11T07:53:14.781 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: starting mon.c rank -1 at public addrs [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] at bind addrs [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] mon_data /var/lib/ceph/mon/ceph-c fsid 41bd1096-7012-11ef-bcea-c7b262605968 2024-09-11T07:53:14.781 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: mon.c@-1(???) e0 preinit fsid 41bd1096-7012-11ef-bcea-c7b262605968 2024-09-11T07:53:15.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: mon.c@-1(synchronizing).mds e1 new map 2024-09-11T07:53:15.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-11T07:53:15.218 INFO:journalctl@ceph.mon.c.smithi110.stdout: e1 2024-09-11T07:53:15.218 INFO:journalctl@ceph.mon.c.smithi110.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-11T07:53:15.219 INFO:journalctl@ceph.mon.c.smithi110.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-11T07:53:15.219 INFO:journalctl@ceph.mon.c.smithi110.stdout: legacy client fscid: -1 2024-09-11T07:53:15.219 INFO:journalctl@ceph.mon.c.smithi110.stdout: 2024-09-11T07:53:15.219 INFO:journalctl@ceph.mon.c.smithi110.stdout: No filesystems configured 2024-09-11T07:53:15.219 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-11T07:53:15.219 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-11T07:53:15.219 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-11T07:53:15.219 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-11T07:53:15.219 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-11T07:53:15.219 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-11T07:53:15.219 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-11T07:53:15.219 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T07:53:15.219 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T07:53:15.219 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T07:53:15.220 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:15.220 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: from='client.? 172.21.15.110:0/2982156807' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-11T07:53:15.220 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T07:53:15.220 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:15.220 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:15.220 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:15.220 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:15.220 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:53:15.220 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:53:15.220 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:53:15.220 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:15.220 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T07:53:15.220 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:53:15.220 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi110:172.21.15.110=a;smithi110:[v2:172.21.15.110:3301,v1:172.21.15.110:6790]=c;smithi139:172.21.15.139=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:53:15.221 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: Saving service mon spec with placement smithi110:172.21.15.110=a;smithi110:[v2:172.21.15.110:3301,v1:172.21.15.110:6790]=c;smithi139:172.21.15.139=b;count:3 2024-09-11T07:53:15.221 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: Deploying daemon mon.b on smithi139 2024-09-11T07:53:15.221 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:15.221 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:15.221 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:15.221 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:15.221 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:15.222 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:15.222 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:53:15.222 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: mon.a calling monitor election 2024-09-11T07:53:15.222 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:53:15.222 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:53:15.222 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:15.222 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:53:15.222 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: mon.b calling monitor election 2024-09-11T07:53:15.222 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:53:15.222 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:15.222 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:53:15.222 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:53:15.222 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-11T07:53:15.222 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: monmap e2: 2 mons at {a=[v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0],b=[v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-11T07:53:15.223 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: fsmap 2024-09-11T07:53:15.223 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T07:53:15.223 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: mgrmap e13: y(active, since 39s) 2024-09-11T07:53:15.223 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: overall HEALTH_OK 2024-09-11T07:53:15.223 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:15.223 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:15.223 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:15.223 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T07:53:15.223 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:53:15.223 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: Deploying daemon mon.c on smithi110 2024-09-11T07:53:15.224 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: from='client.? 172.21.15.139:0/1309787194' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T07:53:15.224 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:53:15.224 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:15.224 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 ceph-mon[31709]: mon.c@-1(synchronizing).paxosservice(auth 1..5) refresh upgraded, format 0 -> 3 2024-09-11T07:53:15.224 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 bash[31689]: 6c2b0e04ed670ee58c9b649addec5719a6067843abb9f26435d8f00388bbb113 2024-09-11T07:53:15.224 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:14 smithi110 systemd[1]: Started Ceph mon.c for 41bd1096-7012-11ef-bcea-c7b262605968. 2024-09-11T07:53:20.159 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:19 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:53:20.159 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:19 smithi110 ceph-mon[31709]: mon.a calling monitor election 2024-09-11T07:53:20.159 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:19 smithi110 ceph-mon[31709]: mon.b calling monitor election 2024-09-11T07:53:20.159 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:19 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:53:20.160 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:19 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:53:20.160 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:19 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:53:20.160 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:19 smithi110 ceph-mon[31709]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:20.160 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:19 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:53:20.160 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:19 smithi110 ceph-mon[31709]: mon.c calling monitor election 2024-09-11T07:53:20.160 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:19 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:53:20.160 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:19 smithi110 ceph-mon[31709]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:20.160 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:19 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:53:20.160 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:19 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:53:20.160 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:19 smithi110 ceph-mon[31709]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-11T07:53:20.160 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:19 smithi110 ceph-mon[31709]: monmap e3: 3 mons at {a=[v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0],b=[v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0],c=[v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-11T07:53:20.160 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:19 smithi110 ceph-mon[31709]: fsmap 2024-09-11T07:53:20.160 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:19 smithi110 ceph-mon[31709]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T07:53:20.161 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:19 smithi110 ceph-mon[31709]: mgrmap e13: y(active, since 47s) 2024-09-11T07:53:20.161 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:19 smithi110 ceph-mon[31709]: overall HEALTH_OK 2024-09-11T07:53:20.161 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:19 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:20.161 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:19 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:20.161 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:19 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:20.161 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:19 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:20.161 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:19 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:20.161 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:19 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:20.161 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:19 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:20.161 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:19 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:53:20.161 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:19 smithi110 ceph-mon[25392]: mon.a calling monitor election 2024-09-11T07:53:20.162 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:19 smithi110 ceph-mon[25392]: mon.b calling monitor election 2024-09-11T07:53:20.162 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:19 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:53:20.162 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:19 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:53:20.162 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:19 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:53:20.162 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:19 smithi110 ceph-mon[25392]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:20.162 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:19 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:53:20.162 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:19 smithi110 ceph-mon[25392]: mon.c calling monitor election 2024-09-11T07:53:20.162 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:19 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:53:20.162 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:19 smithi110 ceph-mon[25392]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:20.162 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:19 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:53:20.162 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:19 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:53:20.162 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:19 smithi110 ceph-mon[25392]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-11T07:53:20.163 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:19 smithi110 ceph-mon[25392]: monmap e3: 3 mons at {a=[v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0],b=[v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0],c=[v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-11T07:53:20.163 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:19 smithi110 ceph-mon[25392]: fsmap 2024-09-11T07:53:20.163 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:19 smithi110 ceph-mon[25392]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T07:53:20.163 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:19 smithi110 ceph-mon[25392]: mgrmap e13: y(active, since 47s) 2024-09-11T07:53:20.163 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:19 smithi110 ceph-mon[25392]: overall HEALTH_OK 2024-09-11T07:53:20.163 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:19 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:20.163 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:19 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:20.163 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:19 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:20.163 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:19 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:20.163 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:19 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:20.163 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:19 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:20.163 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:19 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:20.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:19 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:53:20.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:19 smithi139 ceph-mon[27311]: mon.a calling monitor election 2024-09-11T07:53:20.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:19 smithi139 ceph-mon[27311]: mon.b calling monitor election 2024-09-11T07:53:20.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:19 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:53:20.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:19 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:53:20.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:19 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:53:20.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:19 smithi139 ceph-mon[27311]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:20.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:19 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:53:20.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:19 smithi139 ceph-mon[27311]: mon.c calling monitor election 2024-09-11T07:53:20.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:19 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:53:20.250 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:19 smithi139 ceph-mon[27311]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:20.250 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:19 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:53:20.250 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:19 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:53:20.250 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:19 smithi139 ceph-mon[27311]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-11T07:53:20.250 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:19 smithi139 ceph-mon[27311]: monmap e3: 3 mons at {a=[v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0],b=[v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0],c=[v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-11T07:53:20.250 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:19 smithi139 ceph-mon[27311]: fsmap 2024-09-11T07:53:20.250 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:19 smithi139 ceph-mon[27311]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T07:53:20.250 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:19 smithi139 ceph-mon[27311]: mgrmap e13: y(active, since 47s) 2024-09-11T07:53:20.250 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:19 smithi139 ceph-mon[27311]: overall HEALTH_OK 2024-09-11T07:53:20.250 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:19 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:20.250 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:19 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:20.250 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:19 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:20.250 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:19 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:20.251 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:19 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:20.251 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:19 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:20.251 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:19 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:20.354 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-11T07:53:20.354 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":3,"fsid":"41bd1096-7012-11ef-bcea-c7b262605968","modified":"2024-09-11T07:53:14.777929Z","created":"2024-09-11T07:51:39.629440Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:3300","nonce":0},{"type":"v1","addr":"172.21.15.110:6789","nonce":0}]},"addr":"172.21.15.110:6789/0","public_addr":"172.21.15.110:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:3300","nonce":0},{"type":"v1","addr":"172.21.15.139:6789","nonce":0}]},"addr":"172.21.15.139:6789/0","public_addr":"172.21.15.139:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:3301","nonce":0},{"type":"v1","addr":"172.21.15.110:6790","nonce":0}]},"addr":"172.21.15.110:6790/0","public_addr":"172.21.15.110:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-09-11T07:53:20.354 INFO:teuthology.orchestra.run.smithi139.stderr:dumped monmap epoch 3 2024-09-11T07:53:20.954 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:20 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:53:20.954 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:20 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:53:20.954 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:20 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:53:20.954 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:20 smithi139 ceph-mon[27311]: Updating smithi110:/etc/ceph/ceph.conf 2024-09-11T07:53:20.954 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:20 smithi139 ceph-mon[27311]: Updating smithi139:/etc/ceph/ceph.conf 2024-09-11T07:53:20.955 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:20 smithi139 ceph-mon[27311]: Updating smithi110:/var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:53:20.955 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:20 smithi139 ceph-mon[27311]: Updating smithi139:/var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:53:20.955 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:20 smithi139 ceph-mon[27311]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:20.955 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:20 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:20.955 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:20 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:20.955 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:20 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:20.955 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:20 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:20.955 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:20 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:20.955 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:20 smithi139 ceph-mon[27311]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-11T07:53:20.955 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:20 smithi139 ceph-mon[27311]: from='client.? 172.21.15.139:0/778023154' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T07:53:20.955 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:20 smithi139 ceph-mon[27311]: Reconfiguring mon.a (unknown last config time)... 2024-09-11T07:53:20.955 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:20 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T07:53:20.955 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:20 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T07:53:20.955 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:20 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:53:20.956 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:20 smithi139 ceph-mon[27311]: Reconfiguring daemon mon.a on smithi110 2024-09-11T07:53:20.956 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:20 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:53:20.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:20 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:53:20.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:20 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:53:20.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:20 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:53:20.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:20 smithi110 ceph-mon[31709]: Updating smithi110:/etc/ceph/ceph.conf 2024-09-11T07:53:20.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:20 smithi110 ceph-mon[31709]: Updating smithi139:/etc/ceph/ceph.conf 2024-09-11T07:53:20.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:20 smithi110 ceph-mon[31709]: Updating smithi110:/var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:53:20.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:20 smithi110 ceph-mon[31709]: Updating smithi139:/var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:53:20.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:20 smithi110 ceph-mon[31709]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:20.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:20 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:20.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:20 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:20.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:20 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:20.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:20 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:20.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:20 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:20.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:20 smithi110 ceph-mon[31709]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-11T07:53:20.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:20 smithi110 ceph-mon[31709]: from='client.? 172.21.15.139:0/778023154' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T07:53:20.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:20 smithi110 ceph-mon[31709]: Reconfiguring mon.a (unknown last config time)... 2024-09-11T07:53:20.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:20 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T07:53:20.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:20 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T07:53:20.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:20 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:53:20.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:20 smithi110 ceph-mon[31709]: Reconfiguring daemon mon.a on smithi110 2024-09-11T07:53:20.970 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:20 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:53:20.970 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:20 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:53:20.970 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:20 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:53:20.970 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:20 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:53:20.970 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:20 smithi110 ceph-mon[25392]: Updating smithi110:/etc/ceph/ceph.conf 2024-09-11T07:53:20.970 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:20 smithi110 ceph-mon[25392]: Updating smithi139:/etc/ceph/ceph.conf 2024-09-11T07:53:20.970 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:20 smithi110 ceph-mon[25392]: Updating smithi110:/var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:53:20.970 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:20 smithi110 ceph-mon[25392]: Updating smithi139:/var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:53:20.970 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:20 smithi110 ceph-mon[25392]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:20.970 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:20 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:20.970 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:20 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:20.971 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:20 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:20.971 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:20 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:20.971 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:20 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:20.971 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:20 smithi110 ceph-mon[25392]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-11T07:53:20.971 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:20 smithi110 ceph-mon[25392]: from='client.? 172.21.15.139:0/778023154' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T07:53:20.971 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:20 smithi110 ceph-mon[25392]: Reconfiguring mon.a (unknown last config time)... 2024-09-11T07:53:20.971 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:20 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T07:53:20.971 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:20 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T07:53:20.971 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:20 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:53:20.971 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:20 smithi110 ceph-mon[25392]: Reconfiguring daemon mon.a on smithi110 2024-09-11T07:53:20.971 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:20 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:53:20.977 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-11T07:53:20.978 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- ceph config generate-minimal-conf 2024-09-11T07:53:22.196 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:53:23.116 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:22 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:23.117 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:22 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:23.117 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:22 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:23.117 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:22 smithi110 ceph-mon[31709]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:23.117 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:22 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:23.117 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:22 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:23.117 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:22 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:23.117 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:22 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:23.117 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:22 smithi110 ceph-mon[31709]: Reconfiguring mon.c (monmap changed)... 2024-09-11T07:53:23.117 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:22 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T07:53:23.117 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:22 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T07:53:23.117 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:22 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:53:23.118 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:22 smithi110 ceph-mon[31709]: Reconfiguring daemon mon.c on smithi110 2024-09-11T07:53:23.118 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:22 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:23.118 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:22 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:23.118 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:22 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:23.118 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:22 smithi110 ceph-mon[25392]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:23.118 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:22 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:23.118 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:22 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:23.118 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:22 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:23.118 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:22 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:23.118 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:22 smithi110 ceph-mon[25392]: Reconfiguring mon.c (monmap changed)... 2024-09-11T07:53:23.118 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:22 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T07:53:23.118 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:22 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T07:53:23.119 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:22 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:53:23.119 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:22 smithi110 ceph-mon[25392]: Reconfiguring daemon mon.c on smithi110 2024-09-11T07:53:23.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:22 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:23.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:22 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:23.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:22 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:23.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:22 smithi139 ceph-mon[27311]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:23.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:22 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:23.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:22 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:23.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:22 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:23.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:22 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:23.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:22 smithi139 ceph-mon[27311]: Reconfiguring mon.c (monmap changed)... 2024-09-11T07:53:23.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:22 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T07:53:23.250 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:22 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T07:53:23.250 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:22 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:53:23.250 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:22 smithi139 ceph-mon[27311]: Reconfiguring daemon mon.c on smithi110 2024-09-11T07:53:24.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:24 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:24.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:24 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:24.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:24 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:24.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:24 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:24.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:24 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:24.970 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:24 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:24.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:24 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:24.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:24 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:24.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:24 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:25.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:25 smithi110 ceph-mon[25392]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:25.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:25 smithi110 ceph-mon[31709]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:25.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:25 smithi139 ceph-mon[27311]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:26.946 INFO:teuthology.orchestra.run.smithi110.stdout:# minimal ceph.conf for 41bd1096-7012-11ef-bcea-c7b262605968 2024-09-11T07:53:26.946 INFO:teuthology.orchestra.run.smithi110.stdout:[global] 2024-09-11T07:53:26.946 INFO:teuthology.orchestra.run.smithi110.stdout: fsid = 41bd1096-7012-11ef-bcea-c7b262605968 2024-09-11T07:53:26.946 INFO:teuthology.orchestra.run.smithi110.stdout: mon_host = [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] 2024-09-11T07:53:27.755 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:27 smithi110 ceph-mon[25392]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:27.756 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:27 smithi110 ceph-mon[25392]: from='client.? 172.21.15.110:0/3802915898' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:53:27.756 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:27 smithi110 ceph-mon[31709]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:27.757 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:27 smithi110 ceph-mon[31709]: from='client.? 172.21.15.110:0/3802915898' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:53:27.778 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-11T07:53:27.778 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-11T07:53:27.778 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/etc/ceph/ceph.conf 2024-09-11T07:53:27.807 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-11T07:53:27.807 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T07:53:27.878 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-11T07:53:27.878 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/etc/ceph/ceph.conf 2024-09-11T07:53:27.908 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:27 smithi139 ceph-mon[27311]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:27.908 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:27 smithi139 ceph-mon[27311]: from='client.? 172.21.15.110:0/3802915898' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:53:27.914 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-11T07:53:27.914 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T07:53:27.983 INFO:tasks.cephadm:Adding mgr.y on smithi110 2024-09-11T07:53:27.983 INFO:tasks.cephadm:Adding mgr.x on smithi139 2024-09-11T07:53:27.983 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- ceph orch apply mgr '2;smithi110=y;smithi139=x' 2024-09-11T07:53:28.263 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:53:29.248 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:28 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:29.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:28 smithi139 ceph-mon[27311]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:29.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:28 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:29.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:28 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:29.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:28 smithi139 ceph-mon[27311]: Reconfiguring mon.b (monmap changed)... 2024-09-11T07:53:29.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:28 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T07:53:29.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:28 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T07:53:29.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:28 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:53:29.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:28 smithi139 ceph-mon[27311]: Reconfiguring daemon mon.b on smithi139 2024-09-11T07:53:29.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:28 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:29.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:28 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:29.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:28 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:29.357 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:28 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:29.357 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:28 smithi110 ceph-mon[25392]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:29.357 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:28 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:29.357 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:28 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:29.357 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:28 smithi110 ceph-mon[25392]: Reconfiguring mon.b (monmap changed)... 2024-09-11T07:53:29.357 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:28 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T07:53:29.357 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:28 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T07:53:29.357 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:28 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:53:29.357 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:28 smithi110 ceph-mon[25392]: Reconfiguring daemon mon.b on smithi139 2024-09-11T07:53:29.357 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:28 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:29.357 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:28 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:29.358 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:28 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:29.358 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:28 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:29.358 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:28 smithi110 ceph-mon[31709]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:29.358 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:28 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:29.358 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:28 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:29.358 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:28 smithi110 ceph-mon[31709]: Reconfiguring mon.b (monmap changed)... 2024-09-11T07:53:29.358 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:28 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T07:53:29.358 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:28 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T07:53:29.358 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:28 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:53:29.358 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:28 smithi110 ceph-mon[31709]: Reconfiguring daemon mon.b on smithi139 2024-09-11T07:53:29.358 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:28 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:29.359 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:28 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:29.359 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:28 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:30.824 INFO:teuthology.orchestra.run.smithi139.stdout:Scheduled mgr update... 2024-09-11T07:53:31.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:30 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:31.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:30 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:31.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:30 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:31.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:30 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:31.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:30 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:31.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:30 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:53:31.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:30 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:53:31.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:30 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:53:31.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:30 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:31.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:30 smithi110 ceph-mon[25392]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:31.219 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:30 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:31.219 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:30 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:31.219 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:30 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:31.219 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:30 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:31.219 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:30 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:31.219 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:30 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:53:31.219 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:30 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:53:31.220 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:30 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:53:31.220 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:30 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:31.220 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:30 smithi110 ceph-mon[31709]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:31.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:30 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:31.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:30 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:31.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:30 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:31.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:30 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:31.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:30 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:31.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:30 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:53:31.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:30 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:53:31.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:30 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:53:31.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:30 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:31.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:30 smithi139 ceph-mon[27311]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:31.425 DEBUG:teuthology.orchestra.run.smithi139:mgr.x> sudo journalctl -f -n 0 -u ceph-41bd1096-7012-11ef-bcea-c7b262605968@mgr.x.service 2024-09-11T07:53:31.427 INFO:tasks.cephadm:Deploying OSDs... 2024-09-11T07:53:31.427 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-11T07:53:31.428 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/scratch_devs of=/dev/stdout 2024-09-11T07:53:31.447 DEBUG:teuthology.misc:devs=['/dev/nvme4n1', '/dev/nvme3n1', '/dev/nvme2n1', '/dev/nvme1n1'] 2024-09-11T07:53:31.447 DEBUG:teuthology.orchestra.run.smithi110:> stat /dev/nvme4n1 2024-09-11T07:53:31.505 INFO:teuthology.orchestra.run.smithi110.stdout: File: /dev/nvme4n1 2024-09-11T07:53:31.506 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-11T07:53:31.506 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 5h/5d Inode: 1039 Links: 1 Device type: 103,8 2024-09-11T07:53:31.506 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-11T07:53:31.506 INFO:teuthology.orchestra.run.smithi110.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-11T07:53:31.506 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 2024-09-11 07:53:29.006834600 +0000 2024-09-11T07:53:31.506 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2024-09-11 07:48:36.734618101 +0000 2024-09-11T07:53:31.506 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2024-09-11 07:48:36.734618101 +0000 2024-09-11T07:53:31.506 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: 2024-09-11 07:48:36.732618134 +0000 2024-09-11T07:53:31.506 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/dev/nvme4n1 of=/dev/null count=1 2024-09-11T07:53:31.573 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2024-09-11T07:53:31.573 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2024-09-11T07:53:31.573 INFO:teuthology.orchestra.run.smithi110.stderr:512 bytes copied, 0.000114765 s, 4.5 MB/s 2024-09-11T07:53:31.574 DEBUG:teuthology.orchestra.run.smithi110:> ! mount | grep -v devtmpfs | grep -q /dev/nvme4n1 2024-09-11T07:53:31.630 DEBUG:teuthology.orchestra.run.smithi110:> stat /dev/nvme3n1 2024-09-11T07:53:31.685 INFO:teuthology.orchestra.run.smithi110.stdout: File: /dev/nvme3n1 2024-09-11T07:53:31.685 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-11T07:53:31.685 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 5h/5d Inode: 1028 Links: 1 Device type: 103,6 2024-09-11T07:53:31.686 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-11T07:53:31.686 INFO:teuthology.orchestra.run.smithi110.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-11T07:53:31.686 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 2024-09-11 07:53:28.998834731 +0000 2024-09-11T07:53:31.686 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2024-09-11 07:48:36.599620298 +0000 2024-09-11T07:53:31.686 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2024-09-11 07:48:36.599620298 +0000 2024-09-11T07:53:31.686 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: 2024-09-11 07:48:36.597620331 +0000 2024-09-11T07:53:31.686 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/dev/nvme3n1 of=/dev/null count=1 2024-09-11T07:53:31.750 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2024-09-11T07:53:31.750 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2024-09-11T07:53:31.750 INFO:teuthology.orchestra.run.smithi110.stderr:512 bytes copied, 0.000114183 s, 4.5 MB/s 2024-09-11T07:53:31.751 DEBUG:teuthology.orchestra.run.smithi110:> ! mount | grep -v devtmpfs | grep -q /dev/nvme3n1 2024-09-11T07:53:31.807 DEBUG:teuthology.orchestra.run.smithi110:> stat /dev/nvme2n1 2024-09-11T07:53:31.862 INFO:teuthology.orchestra.run.smithi110.stdout: File: /dev/nvme2n1 2024-09-11T07:53:31.862 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-11T07:53:31.862 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 5h/5d Inode: 1016 Links: 1 Device type: 103,4 2024-09-11T07:53:31.863 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-11T07:53:31.863 INFO:teuthology.orchestra.run.smithi110.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-11T07:53:31.863 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 2024-09-11 07:53:28.990834862 +0000 2024-09-11T07:53:31.863 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2024-09-11 07:48:36.462622528 +0000 2024-09-11T07:53:31.863 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2024-09-11 07:48:36.462622528 +0000 2024-09-11T07:53:31.863 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: 2024-09-11 07:48:36.459622577 +0000 2024-09-11T07:53:31.863 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/dev/nvme2n1 of=/dev/null count=1 2024-09-11T07:53:31.927 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2024-09-11T07:53:31.927 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2024-09-11T07:53:31.927 INFO:teuthology.orchestra.run.smithi110.stderr:512 bytes copied, 0.000137022 s, 3.7 MB/s 2024-09-11T07:53:31.928 DEBUG:teuthology.orchestra.run.smithi110:> ! mount | grep -v devtmpfs | grep -q /dev/nvme2n1 2024-09-11T07:53:31.962 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:31 smithi139 ceph-mon[27311]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi110=y;smithi139=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:53:31.962 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:31 smithi139 ceph-mon[27311]: Saving service mgr spec with placement smithi110=y;smithi139=x;count:2 2024-09-11T07:53:31.962 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:31 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:31.962 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:31 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:53:31.962 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:31 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:53:31.962 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:31 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:53:31.962 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:31 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:31.962 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:31 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T07:53:31.962 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:31 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T07:53:31.962 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:31 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T07:53:31.963 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:31 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:53:31.963 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:31 smithi139 ceph-mon[27311]: Deploying daemon mgr.x on smithi139 2024-09-11T07:53:31.989 DEBUG:teuthology.orchestra.run.smithi110:> stat /dev/nvme1n1 2024-09-11T07:53:32.046 INFO:teuthology.orchestra.run.smithi110.stdout: File: /dev/nvme1n1 2024-09-11T07:53:32.046 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-11T07:53:32.046 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 5h/5d Inode: 1004 Links: 1 Device type: 103,2 2024-09-11T07:53:32.046 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-11T07:53:32.046 INFO:teuthology.orchestra.run.smithi110.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-11T07:53:32.046 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 2024-09-11 07:53:28.982834993 +0000 2024-09-11T07:53:32.046 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2024-09-11 07:48:36.315624921 +0000 2024-09-11T07:53:32.047 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2024-09-11 07:48:36.315624921 +0000 2024-09-11T07:53:32.047 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: 2024-09-11 07:48:36.315624921 +0000 2024-09-11T07:53:32.047 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/dev/nvme1n1 of=/dev/null count=1 2024-09-11T07:53:32.107 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:31 smithi110 ceph-mon[25392]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi110=y;smithi139=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:53:32.107 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:31 smithi110 ceph-mon[25392]: Saving service mgr spec with placement smithi110=y;smithi139=x;count:2 2024-09-11T07:53:32.108 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:31 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:32.108 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:31 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:53:32.108 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:31 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:53:32.108 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:31 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:53:32.108 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:31 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:32.108 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:31 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T07:53:32.108 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:31 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T07:53:32.108 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:31 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T07:53:32.108 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:31 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:53:32.108 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:31 smithi110 ceph-mon[25392]: Deploying daemon mgr.x on smithi139 2024-09-11T07:53:32.109 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:31 smithi110 ceph-mon[31709]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi110=y;smithi139=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:53:32.109 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:31 smithi110 ceph-mon[31709]: Saving service mgr spec with placement smithi110=y;smithi139=x;count:2 2024-09-11T07:53:32.109 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:31 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:32.109 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:31 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:53:32.109 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:31 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:53:32.109 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:31 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:53:32.109 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:31 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:32.109 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:31 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T07:53:32.109 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:31 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T07:53:32.109 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:31 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T07:53:32.109 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:31 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:53:32.109 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:31 smithi110 ceph-mon[31709]: Deploying daemon mgr.x on smithi139 2024-09-11T07:53:32.110 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2024-09-11T07:53:32.110 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2024-09-11T07:53:32.110 INFO:teuthology.orchestra.run.smithi110.stderr:512 bytes copied, 0.000108683 s, 4.7 MB/s 2024-09-11T07:53:32.112 DEBUG:teuthology.orchestra.run.smithi110:> ! mount | grep -v devtmpfs | grep -q /dev/nvme1n1 2024-09-11T07:53:32.168 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-11T07:53:32.168 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/scratch_devs of=/dev/stdout 2024-09-11T07:53:32.184 DEBUG:teuthology.misc:devs=['/dev/nvme4n1', '/dev/nvme3n1', '/dev/nvme2n1', '/dev/nvme1n1'] 2024-09-11T07:53:32.185 DEBUG:teuthology.orchestra.run.smithi139:> stat /dev/nvme4n1 2024-09-11T07:53:32.242 INFO:teuthology.orchestra.run.smithi139.stdout: File: /dev/nvme4n1 2024-09-11T07:53:32.242 INFO:teuthology.orchestra.run.smithi139.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-11T07:53:32.242 INFO:teuthology.orchestra.run.smithi139.stdout:Device: 5h/5d Inode: 1039 Links: 1 Device type: 103,8 2024-09-11T07:53:32.242 INFO:teuthology.orchestra.run.smithi139.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-11T07:53:32.242 INFO:teuthology.orchestra.run.smithi139.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-11T07:53:32.242 INFO:teuthology.orchestra.run.smithi139.stdout:Access: 2024-09-11 07:53:22.917273852 +0000 2024-09-11T07:53:32.242 INFO:teuthology.orchestra.run.smithi139.stdout:Modify: 2024-09-11 07:48:38.356481560 +0000 2024-09-11T07:53:32.242 INFO:teuthology.orchestra.run.smithi139.stdout:Change: 2024-09-11 07:48:38.356481560 +0000 2024-09-11T07:53:32.242 INFO:teuthology.orchestra.run.smithi139.stdout: Birth: 2024-09-11 07:48:38.351481667 +0000 2024-09-11T07:53:32.243 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/dev/nvme4n1 of=/dev/null count=1 2024-09-11T07:53:32.306 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records in 2024-09-11T07:53:32.307 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records out 2024-09-11T07:53:32.307 INFO:teuthology.orchestra.run.smithi139.stderr:512 bytes copied, 0.000219016 s, 2.3 MB/s 2024-09-11T07:53:32.308 DEBUG:teuthology.orchestra.run.smithi139:> ! mount | grep -v devtmpfs | grep -q /dev/nvme4n1 2024-09-11T07:53:32.364 DEBUG:teuthology.orchestra.run.smithi139:> stat /dev/nvme3n1 2024-09-11T07:53:32.419 INFO:teuthology.orchestra.run.smithi139.stdout: File: /dev/nvme3n1 2024-09-11T07:53:32.419 INFO:teuthology.orchestra.run.smithi139.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-11T07:53:32.419 INFO:teuthology.orchestra.run.smithi139.stdout:Device: 5h/5d Inode: 1027 Links: 1 Device type: 103,6 2024-09-11T07:53:32.419 INFO:teuthology.orchestra.run.smithi139.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-11T07:53:32.419 INFO:teuthology.orchestra.run.smithi139.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-11T07:53:32.419 INFO:teuthology.orchestra.run.smithi139.stdout:Access: 2024-09-11 07:53:22.907274071 +0000 2024-09-11T07:53:32.419 INFO:teuthology.orchestra.run.smithi139.stdout:Modify: 2024-09-11 07:48:38.221484443 +0000 2024-09-11T07:53:32.419 INFO:teuthology.orchestra.run.smithi139.stdout:Change: 2024-09-11 07:48:38.221484443 +0000 2024-09-11T07:53:32.420 INFO:teuthology.orchestra.run.smithi139.stdout: Birth: 2024-09-11 07:48:38.215484571 +0000 2024-09-11T07:53:32.420 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/dev/nvme3n1 of=/dev/null count=1 2024-09-11T07:53:32.482 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records in 2024-09-11T07:53:32.482 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records out 2024-09-11T07:53:32.482 INFO:teuthology.orchestra.run.smithi139.stderr:512 bytes copied, 0.000199627 s, 2.6 MB/s 2024-09-11T07:53:32.484 DEBUG:teuthology.orchestra.run.smithi139:> ! mount | grep -v devtmpfs | grep -q /dev/nvme3n1 2024-09-11T07:53:32.540 DEBUG:teuthology.orchestra.run.smithi139:> stat /dev/nvme2n1 2024-09-11T07:53:32.595 INFO:teuthology.orchestra.run.smithi139.stdout: File: /dev/nvme2n1 2024-09-11T07:53:32.595 INFO:teuthology.orchestra.run.smithi139.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-11T07:53:32.595 INFO:teuthology.orchestra.run.smithi139.stdout:Device: 5h/5d Inode: 1015 Links: 1 Device type: 103,4 2024-09-11T07:53:32.596 INFO:teuthology.orchestra.run.smithi139.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-11T07:53:32.596 INFO:teuthology.orchestra.run.smithi139.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-11T07:53:32.596 INFO:teuthology.orchestra.run.smithi139.stdout:Access: 2024-09-11 07:53:22.897274291 +0000 2024-09-11T07:53:32.596 INFO:teuthology.orchestra.run.smithi139.stdout:Modify: 2024-09-11 07:48:38.078487496 +0000 2024-09-11T07:53:32.596 INFO:teuthology.orchestra.run.smithi139.stdout:Change: 2024-09-11 07:48:38.078487496 +0000 2024-09-11T07:53:32.596 INFO:teuthology.orchestra.run.smithi139.stdout: Birth: 2024-09-11 07:48:38.072487624 +0000 2024-09-11T07:53:32.596 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/dev/nvme2n1 of=/dev/null count=1 2024-09-11T07:53:32.807 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records in 2024-09-11T07:53:32.807 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records out 2024-09-11T07:53:32.807 INFO:teuthology.orchestra.run.smithi139.stderr:512 bytes copied, 0.000242884 s, 2.1 MB/s 2024-09-11T07:53:32.808 DEBUG:teuthology.orchestra.run.smithi139:> ! mount | grep -v devtmpfs | grep -q /dev/nvme2n1 2024-09-11T07:53:32.824 DEBUG:teuthology.orchestra.run.smithi139:> stat /dev/nvme1n1 2024-09-11T07:53:32.886 INFO:teuthology.orchestra.run.smithi139.stdout: File: /dev/nvme1n1 2024-09-11T07:53:32.886 INFO:teuthology.orchestra.run.smithi139.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-11T07:53:32.886 INFO:teuthology.orchestra.run.smithi139.stdout:Device: 5h/5d Inode: 1004 Links: 1 Device type: 103,2 2024-09-11T07:53:32.886 INFO:teuthology.orchestra.run.smithi139.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-11T07:53:32.886 INFO:teuthology.orchestra.run.smithi139.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-11T07:53:32.886 INFO:teuthology.orchestra.run.smithi139.stdout:Access: 2024-09-11 07:53:22.886274532 +0000 2024-09-11T07:53:32.886 INFO:teuthology.orchestra.run.smithi139.stdout:Modify: 2024-09-11 07:48:37.934490571 +0000 2024-09-11T07:53:32.886 INFO:teuthology.orchestra.run.smithi139.stdout:Change: 2024-09-11 07:48:37.934490571 +0000 2024-09-11T07:53:32.886 INFO:teuthology.orchestra.run.smithi139.stdout: Birth: 2024-09-11 07:48:37.929490677 +0000 2024-09-11T07:53:32.886 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/dev/nvme1n1 of=/dev/null count=1 2024-09-11T07:53:32.909 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:32 smithi139 ceph-mon[27311]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:32.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:32 smithi110 ceph-mon[25392]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:32.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:32 smithi110 ceph-mon[31709]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:32.998 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records in 2024-09-11T07:53:32.998 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records out 2024-09-11T07:53:32.998 INFO:teuthology.orchestra.run.smithi139.stderr:512 bytes copied, 0.000284509 s, 1.8 MB/s 2024-09-11T07:53:33.000 DEBUG:teuthology.orchestra.run.smithi139:> ! mount | grep -v devtmpfs | grep -q /dev/nvme1n1 2024-09-11T07:53:33.015 INFO:tasks.cephadm:Deploying osd.0 on smithi110 with /dev/nvme1n1... 2024-09-11T07:53:33.016 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/nvme1n1 2024-09-11T07:53:33.225 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:53:33 smithi139 systemd[1]: Starting Ceph mgr.x for 41bd1096-7012-11ef-bcea-c7b262605968... 2024-09-11T07:53:33.250 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:53:33.499 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:53:33 smithi139 podman[29505]: 2024-09-11 07:53:33.224234038 +0000 UTC m=+0.106982580 container create 61a18e2cfef24f75da2294ba4d3b4c7eaa37089343ed1e3e09b9b37787eb1a60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T07:53:33.499 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:53:33 smithi139 podman[29505]: 2024-09-11 07:53:33.132941269 +0000 UTC m=+0.015689812 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T07:53:33.499 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:53:33 smithi139 podman[29505]: 2024-09-11 07:53:33.43484839 +0000 UTC m=+0.317596933 container init 61a18e2cfef24f75da2294ba4d3b4c7eaa37089343ed1e3e09b9b37787eb1a60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-11T07:53:33.499 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:53:33 smithi139 podman[29505]: 2024-09-11 07:53:33.437690242 +0000 UTC m=+0.320438794 container start 61a18e2cfef24f75da2294ba4d3b4c7eaa37089343ed1e3e09b9b37787eb1a60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-11T07:53:33.499 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:53:33 smithi139 ceph-mgr[29524]: -- 172.21.15.139:0/858632029 <== mon.1 v2:172.21.15.139:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5580a9826340 con 0x5580a9833000 2024-09-11T07:53:33.809 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:53:33 smithi139 bash[29505]: 61a18e2cfef24f75da2294ba4d3b4c7eaa37089343ed1e3e09b9b37787eb1a60 2024-09-11T07:53:33.812 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:53:33 smithi139 systemd[1]: Started Ceph mgr.x for 41bd1096-7012-11ef-bcea-c7b262605968. 2024-09-11T07:53:33.812 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:53:33 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x[29520]: 2024-09-11T07:53:33.551+0000 7f0b45ff6ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T07:53:33.812 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:53:33 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x[29520]: 2024-09-11T07:53:33.610+0000 7f0b45ff6ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T07:53:33.812 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:53:33 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x[29520]: 2024-09-11T07:53:33.679+0000 7f0b45ff6ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T07:53:33.812 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:53:33 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x[29520]: 2024-09-11T07:53:33.807+0000 7f0b45ff6ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T07:53:34.200 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:53:33 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x[29520]: 2024-09-11T07:53:33.942+0000 7f0b45ff6ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T07:53:34.201 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:53:34 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x[29520]: 2024-09-11T07:53:34.199+0000 7f0b45ff6ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T07:53:34.499 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:53:34 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x[29520]: 2024-09-11T07:53:34.258+0000 7f0b45ff6ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T07:53:34.499 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:53:34 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x[29520]: 2024-09-11T07:53:34.420+0000 7f0b45ff6ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T07:53:34.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:34 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:34.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:34 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:34.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:34 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:34.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:34 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:34.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:34 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:53:34.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:34 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:53:34.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:34 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:53:34.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:34 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:34.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:34 smithi110 ceph-mon[25392]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-11T07:53:34.720 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:34 smithi110 ceph-mon[25392]: Reconfiguring mgr.y (unknown last config time)... 2024-09-11T07:53:34.720 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:34 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T07:53:34.720 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:34 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T07:53:34.720 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:34 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:53:34.720 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:34 smithi110 ceph-mon[25392]: Reconfiguring daemon mgr.y on smithi110 2024-09-11T07:53:34.721 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:34 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:34.721 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:34 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:34.721 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:34 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:34.721 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:34 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:34.721 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:34 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:53:34.721 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:34 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:53:34.722 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:34 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:53:34.722 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:34 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:34.722 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:34 smithi110 ceph-mon[31709]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-11T07:53:34.722 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:34 smithi110 ceph-mon[31709]: Reconfiguring mgr.y (unknown last config time)... 2024-09-11T07:53:34.722 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:34 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T07:53:34.722 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:34 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T07:53:34.723 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:34 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:53:34.723 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:34 smithi110 ceph-mon[31709]: Reconfiguring daemon mgr.y on smithi110 2024-09-11T07:53:34.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:34 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:34.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:34 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:34.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:34 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:34.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:34 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:34.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:34 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:53:34.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:34 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:53:34.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:34 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:53:34.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:34 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:34.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:34 smithi139 ceph-mon[27311]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-11T07:53:34.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:34 smithi139 ceph-mon[27311]: Reconfiguring mgr.y (unknown last config time)... 2024-09-11T07:53:34.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:34 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T07:53:35.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:34 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T07:53:35.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:34 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:53:35.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:34 smithi139 ceph-mon[27311]: Reconfiguring daemon mgr.y on smithi110 2024-09-11T07:53:35.338 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:53:35 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x[29520]: 2024-09-11T07:53:35.283+0000 7f0b45ff6ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T07:53:35.414 INFO:teuthology.orchestra.run.smithi110.stderr:--> Zapping: /dev/nvme1n1 2024-09-11T07:53:35.414 INFO:teuthology.orchestra.run.smithi110.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-09-11T07:53:35.417 INFO:teuthology.orchestra.run.smithi110.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme1n1 bs=1M count=10 conv=fsync 2024-09-11T07:53:35.428 INFO:teuthology.orchestra.run.smithi110.stderr: stderr: 10+0 records in 2024-09-11T07:53:35.428 INFO:teuthology.orchestra.run.smithi110.stderr:10+0 records out 2024-09-11T07:53:35.428 INFO:teuthology.orchestra.run.smithi110.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00967724 s, 1.1 GB/s 2024-09-11T07:53:35.428 INFO:teuthology.orchestra.run.smithi110.stderr:--> Zapping successful for: 2024-09-11T07:53:35.689 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:53:35 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x[29520]: 2024-09-11T07:53:35.350+0000 7f0b45ff6ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T07:53:35.942 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:35 smithi139 ceph-mon[27311]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:35.956 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:35 smithi110 ceph-mon[25392]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:35.956 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:35 smithi110 ceph-mon[31709]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:36.205 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:53:36 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x[29520]: 2024-09-11T07:53:36.091+0000 7f0b45ff6ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T07:53:36.499 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:53:36 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x[29520]: 2024-09-11T07:53:36.208+0000 7f0b45ff6ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T07:53:36.499 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:53:36 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x[29520]: 2024-09-11T07:53:36.334+0000 7f0b45ff6ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T07:53:36.499 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:53:36 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x[29520]: 2024-09-11T07:53:36.396+0000 7f0b45ff6ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T07:53:36.614 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi110:/dev/nvme1n1 2024-09-11T07:53:36.795 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:53:36 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x[29520]: 2024-09-11T07:53:36.537+0000 7f0b45ff6ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T07:53:36.796 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:53:36 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x[29520]: 2024-09-11T07:53:36.626+0000 7f0b45ff6ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T07:53:36.796 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:53:36 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x[29520]: 2024-09-11T07:53:36.697+0000 7f0b45ff6ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T07:53:36.850 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:53:37.061 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:53:36 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x[29520]: 2024-09-11T07:53:36.841+0000 7f0b45ff6ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T07:53:37.061 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:53:36 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x[29520]: 2024-09-11T07:53:36.908+0000 7f0b45ff6ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T07:53:37.061 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:53:36 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x[29520]: 2024-09-11T07:53:36.977+0000 7f0b45ff6ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T07:53:37.061 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:53:37 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x[29520]: 2024-09-11T07:53:37.059+0000 7f0b45ff6ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T07:53:37.498 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:53:37 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x[29520]: 2024-09-11T07:53:37.117+0000 7f0b45ff6ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T07:53:37.499 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:53:37 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x[29520]: 2024-09-11T07:53:37.177+0000 7f0b45ff6ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T07:53:37.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:37 smithi139 ceph-mon[27311]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:37.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:37 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:37.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:37 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:37.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:37 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:53:37.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:37 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:53:37.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:37 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:53:37.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:37 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:37.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:37 smithi139 ceph-mon[27311]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-11T07:53:37.500 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:37 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:37.581 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:37 smithi110 ceph-mon[25392]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:37.581 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:37 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:37.581 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:37 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:37.581 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:37 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:53:37.581 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:37 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:53:37.581 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:37 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:53:37.581 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:37 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:37.581 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:37 smithi110 ceph-mon[25392]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-11T07:53:37.581 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:37 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:37.582 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:37 smithi110 ceph-mon[31709]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:37.582 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:37 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:37.582 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:37 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:37.582 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:37 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:53:37.582 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:37 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:53:37.582 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:37 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:53:37.582 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:37 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:37.582 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:37 smithi110 ceph-mon[31709]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-11T07:53:37.582 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:37 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:38.248 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:53:37 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x[29520]: 2024-09-11T07:53:37.942+0000 7f0b45ff6ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T07:53:38.958 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:38 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:38.959 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:38 smithi110 ceph-mon[31709]: Standby manager daemon x started 2024-09-11T07:53:38.959 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:38 smithi110 ceph-mon[31709]: from='mgr.? 172.21.15.139:0/207556290' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-11T07:53:38.959 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:38 smithi110 ceph-mon[31709]: from='mgr.? 172.21.15.139:0/207556290' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T07:53:38.959 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:38 smithi110 ceph-mon[31709]: from='mgr.? 172.21.15.139:0/207556290' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-11T07:53:38.959 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:38 smithi110 ceph-mon[31709]: from='mgr.? 172.21.15.139:0/207556290' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T07:53:38.959 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:38 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:38.959 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:38 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:38.959 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:38 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:53:38.959 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:38 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:38.959 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:38 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:53:38.959 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:38 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:53:38.959 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:38 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:38.959 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:38 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:53:38.960 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:38 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:53:38.960 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:38 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:53:38.960 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:38 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:38.960 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:38 smithi110 ceph-mon[25392]: Standby manager daemon x started 2024-09-11T07:53:38.960 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:38 smithi110 ceph-mon[25392]: from='mgr.? 172.21.15.139:0/207556290' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-11T07:53:38.960 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:38 smithi110 ceph-mon[25392]: from='mgr.? 172.21.15.139:0/207556290' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T07:53:38.960 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:38 smithi110 ceph-mon[25392]: from='mgr.? 172.21.15.139:0/207556290' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-11T07:53:38.960 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:38 smithi110 ceph-mon[25392]: from='mgr.? 172.21.15.139:0/207556290' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T07:53:38.960 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:38 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:38.960 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:38 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:38.960 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:38 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:53:38.961 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:38 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:38.961 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:38 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:53:38.961 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:38 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:53:38.961 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:38 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:38.961 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:38 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:53:38.961 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:38 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:53:38.961 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:38 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:53:38.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:38 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:38.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:38 smithi139 ceph-mon[27311]: Standby manager daemon x started 2024-09-11T07:53:38.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:38 smithi139 ceph-mon[27311]: from='mgr.? 172.21.15.139:0/207556290' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-11T07:53:38.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:38 smithi139 ceph-mon[27311]: from='mgr.? 172.21.15.139:0/207556290' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T07:53:38.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:38 smithi139 ceph-mon[27311]: from='mgr.? 172.21.15.139:0/207556290' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-11T07:53:38.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:38 smithi139 ceph-mon[27311]: from='mgr.? 172.21.15.139:0/207556290' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T07:53:38.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:38 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:38.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:38 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:38.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:38 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:53:38.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:38 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:38.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:38 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:53:39.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:38 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:53:39.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:38 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:39.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:38 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:53:39.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:38 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:53:39.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:38 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:53:39.917 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:39 smithi110 ceph-mon[25392]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:39.917 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:39 smithi110 ceph-mon[25392]: from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:53:39.917 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:39 smithi110 ceph-mon[25392]: mgrmap e14: y(active, since 66s), standbys: x 2024-09-11T07:53:39.917 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:39 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T07:53:39.917 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:39 smithi110 ceph-mon[31709]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:39.917 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:39 smithi110 ceph-mon[31709]: from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:53:39.917 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:39 smithi110 ceph-mon[31709]: mgrmap e14: y(active, since 66s), standbys: x 2024-09-11T07:53:39.918 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:39 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T07:53:39.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:39 smithi139 ceph-mon[27311]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:39.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:39 smithi139 ceph-mon[27311]: from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:53:39.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:39 smithi139 ceph-mon[27311]: mgrmap e14: y(active, since 66s), standbys: x 2024-09-11T07:53:39.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:39 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T07:53:41.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:41 smithi110 ceph-mon[25392]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:41.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:41 smithi110 ceph-mon[25392]: from='client.? 172.21.15.110:0/3997508317' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "05f2cdd1-77dc-4b26-afa8-16099c528c10"}]: dispatch 2024-09-11T07:53:41.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:41 smithi110 ceph-mon[25392]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "05f2cdd1-77dc-4b26-afa8-16099c528c10"}]: dispatch 2024-09-11T07:53:41.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:41 smithi110 ceph-mon[25392]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "05f2cdd1-77dc-4b26-afa8-16099c528c10"}]': finished 2024-09-11T07:53:41.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:41 smithi110 ceph-mon[25392]: osdmap e5: 1 total, 0 up, 1 in 2024-09-11T07:53:41.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:41 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:53:41.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:41 smithi110 ceph-mon[31709]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:41.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:41 smithi110 ceph-mon[31709]: from='client.? 172.21.15.110:0/3997508317' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "05f2cdd1-77dc-4b26-afa8-16099c528c10"}]: dispatch 2024-09-11T07:53:41.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:41 smithi110 ceph-mon[31709]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "05f2cdd1-77dc-4b26-afa8-16099c528c10"}]: dispatch 2024-09-11T07:53:41.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:41 smithi110 ceph-mon[31709]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "05f2cdd1-77dc-4b26-afa8-16099c528c10"}]': finished 2024-09-11T07:53:41.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:41 smithi110 ceph-mon[31709]: osdmap e5: 1 total, 0 up, 1 in 2024-09-11T07:53:41.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:41 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:53:41.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:41 smithi139 ceph-mon[27311]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:41.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:41 smithi139 ceph-mon[27311]: from='client.? 172.21.15.110:0/3997508317' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "05f2cdd1-77dc-4b26-afa8-16099c528c10"}]: dispatch 2024-09-11T07:53:41.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:41 smithi139 ceph-mon[27311]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "05f2cdd1-77dc-4b26-afa8-16099c528c10"}]: dispatch 2024-09-11T07:53:41.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:41 smithi139 ceph-mon[27311]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "05f2cdd1-77dc-4b26-afa8-16099c528c10"}]': finished 2024-09-11T07:53:41.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:41 smithi139 ceph-mon[27311]: osdmap e5: 1 total, 0 up, 1 in 2024-09-11T07:53:41.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:41 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:53:42.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:42 smithi110 ceph-mon[25392]: from='client.? 172.21.15.110:0/3780082751' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:53:42.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:42 smithi110 ceph-mon[31709]: from='client.? 172.21.15.110:0/3780082751' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:53:42.998 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:42 smithi139 ceph-mon[27311]: from='client.? 172.21.15.110:0/3780082751' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:53:43.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:43 smithi110 ceph-mon[25392]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:43.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:43 smithi110 ceph-mon[31709]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:43.998 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:43 smithi139 ceph-mon[27311]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:45.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:45 smithi110 ceph-mon[25392]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:45.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:45 smithi110 ceph-mon[31709]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:45.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:45 smithi139 ceph-mon[27311]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:47.917 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:47 smithi110 ceph-mon[25392]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:47.917 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:47 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-11T07:53:47.917 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:47 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:53:47.917 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:47 smithi110 ceph-mon[31709]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:47.917 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:47 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-11T07:53:47.917 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:47 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:53:47.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:47 smithi139 ceph-mon[27311]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:47.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:47 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-11T07:53:47.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:47 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:53:48.834 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:48 smithi110 ceph-mon[25392]: Deploying daemon osd.0 on smithi110 2024-09-11T07:53:48.835 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:48 smithi110 ceph-mon[31709]: Deploying daemon osd.0 on smithi110 2024-09-11T07:53:48.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:48 smithi139 ceph-mon[27311]: Deploying daemon osd.0 on smithi110 2024-09-11T07:53:49.822 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:49 smithi110 ceph-mon[25392]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:49.822 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:49 smithi110 ceph-mon[31709]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:49.998 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:49 smithi139 ceph-mon[27311]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:50.960 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:50 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:50.960 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:50 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:50.961 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:50 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:50.961 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:50 smithi110 ceph-mon[25392]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:50.961 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:50 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:50.961 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:50 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:50.961 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:50 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:50.961 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:50 smithi110 ceph-mon[31709]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:51.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:50 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:51.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:50 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:51.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:50 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:51.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:50 smithi139 ceph-mon[27311]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:53.248 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:53 smithi139 ceph-mon[27311]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:53.462 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:53 smithi110 ceph-mon[25392]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:53.463 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:53 smithi110 ceph-mon[31709]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:54.422 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:54 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:53:54.423 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:54 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:54.423 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:54 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:53:54.423 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:54 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:53:54.423 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:54 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:54.423 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:54 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:54.423 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:54 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:53:54.423 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:54 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:54.423 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:54 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:53:54.423 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:54 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:53:54.423 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:54 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:54.424 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:54 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:54.501 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:54 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:53:54.501 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:54 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:54.502 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:54 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:53:54.502 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:54 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:53:54.502 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:54 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:54.502 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:54 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:55.377 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:55 smithi110 ceph-mon[31709]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:55.377 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:55 smithi110 ceph-mon[25392]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:55.498 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:55 smithi139 ceph-mon[27311]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:56.717 INFO:teuthology.orchestra.run.smithi110.stdout:Created osd(s) 0 on host 'smithi110' 2024-09-11T07:53:57.329 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:57 smithi110 ceph-mon[25392]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:57.329 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:57 smithi110 ceph-mon[25392]: from='osd.0 [v2:172.21.15.110:6802/2716989082,v1:172.21.15.110:6803/2716989082]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T07:53:57.330 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:57 smithi110 ceph-mon[25392]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T07:53:57.330 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:57 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:53:57.330 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:57 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:57.330 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:57 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:53:57.330 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:57 smithi110 ceph-mon[25392]: Adjusting osd_memory_target on smithi110 to 15040M 2024-09-11T07:53:57.330 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:57 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:57.330 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:57 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:57.330 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:57 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:53:57.330 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:57 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:53:57.330 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:57 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:57.330 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:57 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:57.331 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:57 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:57.331 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:57 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:57.331 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:57 smithi110 ceph-mon[31709]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:57.331 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:57 smithi110 ceph-mon[31709]: from='osd.0 [v2:172.21.15.110:6802/2716989082,v1:172.21.15.110:6803/2716989082]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T07:53:57.331 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:57 smithi110 ceph-mon[31709]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T07:53:57.331 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:57 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:53:57.331 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:57 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:57.331 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:57 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:53:57.331 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:57 smithi110 ceph-mon[31709]: Adjusting osd_memory_target on smithi110 to 15040M 2024-09-11T07:53:57.331 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:57 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:57.331 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:57 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:57.331 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:57 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:53:57.332 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:57 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:53:57.332 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:57 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:57.332 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:57 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:57.332 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:57 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:57.332 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:57 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:57.353 DEBUG:teuthology.orchestra.run.smithi110:osd.0> sudo journalctl -f -n 0 -u ceph-41bd1096-7012-11ef-bcea-c7b262605968@osd.0.service 2024-09-11T07:53:57.356 INFO:tasks.cephadm:Deploying osd.1 on smithi110 with /dev/nvme2n1... 2024-09-11T07:53:57.356 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/nvme2n1 2024-09-11T07:53:57.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:57 smithi139 ceph-mon[27311]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:57.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:57 smithi139 ceph-mon[27311]: from='osd.0 [v2:172.21.15.110:6802/2716989082,v1:172.21.15.110:6803/2716989082]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T07:53:57.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:57 smithi139 ceph-mon[27311]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T07:53:57.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:57 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:53:57.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:57 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:57.500 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:57 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:53:57.500 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:57 smithi139 ceph-mon[27311]: Adjusting osd_memory_target on smithi110 to 15040M 2024-09-11T07:53:57.500 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:57 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:57.500 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:57 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:57.500 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:57 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:53:57.500 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:57 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:53:57.500 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:57 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:57.500 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:57 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:57.500 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:57 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:57.500 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:57 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:53:57.629 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:53:58.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:58 smithi110 ceph-mon[25392]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-11T07:53:58.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:58 smithi110 ceph-mon[25392]: osdmap e6: 1 total, 0 up, 1 in 2024-09-11T07:53:58.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:58 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:53:58.469 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:58 smithi110 ceph-mon[25392]: from='osd.0 [v2:172.21.15.110:6802/2716989082,v1:172.21.15.110:6803/2716989082]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-09-11T07:53:58.469 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:58 smithi110 ceph-mon[25392]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-09-11T07:53:58.469 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:58 smithi110 ceph-mon[31709]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-11T07:53:58.469 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:58 smithi110 ceph-mon[31709]: osdmap e6: 1 total, 0 up, 1 in 2024-09-11T07:53:58.469 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:58 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:53:58.469 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:58 smithi110 ceph-mon[31709]: from='osd.0 [v2:172.21.15.110:6802/2716989082,v1:172.21.15.110:6803/2716989082]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-09-11T07:53:58.469 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:58 smithi110 ceph-mon[31709]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-09-11T07:53:58.470 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 11 07:53:58 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-0[36077]: 2024-09-11T07:53:58.217+0000 7f70e8a52640 -1 osd.0 0 waiting for initial osdmap 2024-09-11T07:53:58.470 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 11 07:53:58 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-0[36077]: 2024-09-11T07:53:58.219+0000 7f70e3889640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T07:53:58.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:58 smithi139 ceph-mon[27311]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-11T07:53:58.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:58 smithi139 ceph-mon[27311]: osdmap e6: 1 total, 0 up, 1 in 2024-09-11T07:53:58.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:58 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:53:58.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:58 smithi139 ceph-mon[27311]: from='osd.0 [v2:172.21.15.110:6802/2716989082,v1:172.21.15.110:6803/2716989082]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-09-11T07:53:58.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:58 smithi139 ceph-mon[27311]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-09-11T07:53:59.459 INFO:teuthology.orchestra.run.smithi110.stderr:--> Zapping: /dev/nvme2n1 2024-09-11T07:53:59.460 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:59 smithi110 ceph-mon[25392]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:59.460 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:59 smithi110 ceph-mon[25392]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-09-11T07:53:59.460 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:59 smithi110 ceph-mon[25392]: osdmap e7: 1 total, 0 up, 1 in 2024-09-11T07:53:59.460 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:59 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:53:59.460 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:53:59 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:53:59.461 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:59 smithi110 ceph-mon[31709]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:59.461 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:59 smithi110 ceph-mon[31709]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-09-11T07:53:59.461 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:59 smithi110 ceph-mon[31709]: osdmap e7: 1 total, 0 up, 1 in 2024-09-11T07:53:59.461 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:59 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:53:59.461 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:53:59 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:53:59.461 INFO:teuthology.orchestra.run.smithi110.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-09-11T07:53:59.462 INFO:teuthology.orchestra.run.smithi110.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme2n1 bs=1M count=10 conv=fsync 2024-09-11T07:53:59.475 INFO:teuthology.orchestra.run.smithi110.stderr: stderr: 10+0 records in 2024-09-11T07:53:59.475 INFO:teuthology.orchestra.run.smithi110.stderr:10+0 records out 2024-09-11T07:53:59.475 INFO:teuthology.orchestra.run.smithi110.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0103675 s, 1.0 GB/s 2024-09-11T07:53:59.475 INFO:teuthology.orchestra.run.smithi110.stderr:--> Zapping successful for: 2024-09-11T07:53:59.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:59 smithi139 ceph-mon[27311]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:53:59.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:59 smithi139 ceph-mon[27311]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-09-11T07:53:59.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:59 smithi139 ceph-mon[27311]: osdmap e7: 1 total, 0 up, 1 in 2024-09-11T07:53:59.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:59 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:53:59.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:53:59 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:54:00.115 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi110:/dev/nvme2n1 2024-09-11T07:54:00.358 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:54:00.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:00 smithi110 ceph-mon[25392]: purged_snaps scrub starts 2024-09-11T07:54:00.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:00 smithi110 ceph-mon[25392]: purged_snaps scrub ok 2024-09-11T07:54:00.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:00 smithi110 ceph-mon[25392]: osd.0 [v2:172.21.15.110:6802/2716989082,v1:172.21.15.110:6803/2716989082] boot 2024-09-11T07:54:00.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:00 smithi110 ceph-mon[25392]: osdmap e8: 1 total, 1 up, 1 in 2024-09-11T07:54:00.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:00 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:54:00.468 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:00 smithi110 ceph-mon[31709]: purged_snaps scrub starts 2024-09-11T07:54:00.469 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:00 smithi110 ceph-mon[31709]: purged_snaps scrub ok 2024-09-11T07:54:00.469 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:00 smithi110 ceph-mon[31709]: osd.0 [v2:172.21.15.110:6802/2716989082,v1:172.21.15.110:6803/2716989082] boot 2024-09-11T07:54:00.469 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:00 smithi110 ceph-mon[31709]: osdmap e8: 1 total, 1 up, 1 in 2024-09-11T07:54:00.469 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:00 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:54:00.498 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:00 smithi139 ceph-mon[27311]: purged_snaps scrub starts 2024-09-11T07:54:00.498 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:00 smithi139 ceph-mon[27311]: purged_snaps scrub ok 2024-09-11T07:54:00.498 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:00 smithi139 ceph-mon[27311]: osd.0 [v2:172.21.15.110:6802/2716989082,v1:172.21.15.110:6803/2716989082] boot 2024-09-11T07:54:00.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:00 smithi139 ceph-mon[27311]: osdmap e8: 1 total, 1 up, 1 in 2024-09-11T07:54:00.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:00 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:54:01.423 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:01 smithi110 ceph-mon[31709]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:54:01.423 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:01 smithi110 ceph-mon[31709]: osdmap e9: 1 total, 1 up, 1 in 2024-09-11T07:54:01.423 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:01 smithi110 ceph-mon[25392]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:54:01.424 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:01 smithi110 ceph-mon[25392]: osdmap e9: 1 total, 1 up, 1 in 2024-09-11T07:54:01.498 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:01 smithi139 ceph-mon[27311]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:54:01.498 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:01 smithi139 ceph-mon[27311]: osdmap e9: 1 total, 1 up, 1 in 2024-09-11T07:54:02.498 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:02 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:54:02.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:02 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:54:02.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:02 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:54:02.521 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:02 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:54:02.521 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:02 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:54:02.521 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:02 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:54:02.522 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:02 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:54:02.522 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:02 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:54:02.522 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:02 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:54:03.468 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:03 smithi110 ceph-mon[31709]: from='client.14238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:/dev/nvme2n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:54:03.468 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:03 smithi110 ceph-mon[31709]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:54:03.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:03 smithi110 ceph-mon[25392]: from='client.14238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:/dev/nvme2n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:54:03.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:03 smithi110 ceph-mon[25392]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:54:03.498 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:03 smithi139 ceph-mon[27311]: from='client.14238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:/dev/nvme2n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:54:03.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:03 smithi139 ceph-mon[27311]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:54:05.319 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:05 smithi110 ceph-mon[25392]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:54:05.319 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:05 smithi110 ceph-mon[25392]: from='client.? 172.21.15.110:0/1926481311' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e01c286-b92c-46b3-b61c-f8aea652b8be"}]: dispatch 2024-09-11T07:54:05.319 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:05 smithi110 ceph-mon[25392]: from='client.? 172.21.15.110:0/1926481311' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1e01c286-b92c-46b3-b61c-f8aea652b8be"}]': finished 2024-09-11T07:54:05.320 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:05 smithi110 ceph-mon[25392]: osdmap e10: 2 total, 1 up, 2 in 2024-09-11T07:54:05.320 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:05 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:54:05.320 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:05 smithi110 ceph-mon[31709]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:54:05.320 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:05 smithi110 ceph-mon[31709]: from='client.? 172.21.15.110:0/1926481311' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e01c286-b92c-46b3-b61c-f8aea652b8be"}]: dispatch 2024-09-11T07:54:05.320 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:05 smithi110 ceph-mon[31709]: from='client.? 172.21.15.110:0/1926481311' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1e01c286-b92c-46b3-b61c-f8aea652b8be"}]': finished 2024-09-11T07:54:05.320 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:05 smithi110 ceph-mon[31709]: osdmap e10: 2 total, 1 up, 2 in 2024-09-11T07:54:05.320 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:05 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:54:05.498 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:05 smithi139 ceph-mon[27311]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:54:05.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:05 smithi139 ceph-mon[27311]: from='client.? 172.21.15.110:0/1926481311' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e01c286-b92c-46b3-b61c-f8aea652b8be"}]: dispatch 2024-09-11T07:54:05.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:05 smithi139 ceph-mon[27311]: from='client.? 172.21.15.110:0/1926481311' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1e01c286-b92c-46b3-b61c-f8aea652b8be"}]': finished 2024-09-11T07:54:05.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:05 smithi139 ceph-mon[27311]: osdmap e10: 2 total, 1 up, 2 in 2024-09-11T07:54:05.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:05 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:54:06.437 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:06 smithi110 ceph-mon[25392]: from='client.? 172.21.15.110:0/3433692607' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:54:06.437 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:06 smithi110 ceph-mon[31709]: from='client.? 172.21.15.110:0/3433692607' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:54:06.498 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:06 smithi139 ceph-mon[27311]: from='client.? 172.21.15.110:0/3433692607' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:54:07.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:07 smithi110 ceph-mon[25392]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:54:07.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:07 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:07.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:07 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:07.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:07 smithi110 ceph-mon[25392]: Detected new or changed devices on smithi110 2024-09-11T07:54:07.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:07 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:07.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:07 smithi110 ceph-mon[31709]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:54:07.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:07 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:07.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:07 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:07.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:07 smithi110 ceph-mon[31709]: Detected new or changed devices on smithi110 2024-09-11T07:54:07.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:07 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:07.748 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:07 smithi139 ceph-mon[27311]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:54:07.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:07 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:07.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:07 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:07.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:07 smithi139 ceph-mon[27311]: Detected new or changed devices on smithi110 2024-09-11T07:54:07.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:07 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:09.690 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:09 smithi110 ceph-mon[25392]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:54:09.690 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:09 smithi110 ceph-mon[31709]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:54:09.748 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:09 smithi139 ceph-mon[27311]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:54:11.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:11 smithi110 ceph-mon[25392]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:54:11.468 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:11 smithi110 ceph-mon[31709]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:54:11.748 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:11 smithi139 ceph-mon[27311]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:54:12.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:12 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-11T07:54:12.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:12 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:54:12.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:12 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-11T07:54:12.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:12 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:54:12.748 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:12 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-11T07:54:12.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:12 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:54:13.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:13 smithi110 ceph-mon[25392]: Deploying daemon osd.1 on smithi110 2024-09-11T07:54:13.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:13 smithi110 ceph-mon[25392]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:54:13.468 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:13 smithi110 ceph-mon[31709]: Deploying daemon osd.1 on smithi110 2024-09-11T07:54:13.469 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:13 smithi110 ceph-mon[31709]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:54:13.709 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:13 smithi139 ceph-mon[27311]: Deploying daemon osd.1 on smithi110 2024-09-11T07:54:13.709 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:13 smithi139 ceph-mon[27311]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:54:15.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:15 smithi110 ceph-mon[25392]: pgmap v51: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:54:15.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:15 smithi110 ceph-mon[31709]: pgmap v51: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:54:15.748 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:15 smithi139 ceph-mon[27311]: pgmap v51: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:54:17.623 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:17 smithi139 ceph-mon[27311]: pgmap v52: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:54:17.623 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:17 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:17.623 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:17 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:17.623 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:17 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:17.655 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:17 smithi110 ceph-mon[25392]: pgmap v52: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:54:17.655 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:17 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:17.655 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:17 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:17.655 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:17 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:17.656 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:17 smithi110 ceph-mon[31709]: pgmap v52: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:54:17.656 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:17 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:17.656 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:17 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:17.656 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:17 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:18.708 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:18 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:54:18.708 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:18 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:18.708 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:18 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:54:18.708 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:18 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:54:18.708 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:18 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:18.708 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:18 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:18.708 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:18 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:54:18.709 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:18 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:18.709 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:18 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:54:18.709 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:18 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:54:18.709 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:18 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:18.709 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:18 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:18.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:18 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:54:18.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:18 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:18.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:18 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:54:18.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:18 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:54:18.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:18 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:18.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:18 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:19.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:19 smithi110 ceph-mon[31709]: pgmap v53: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:54:19.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:19 smithi110 ceph-mon[25392]: pgmap v53: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:54:19.748 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:19 smithi139 ceph-mon[27311]: pgmap v53: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:54:21.163 INFO:teuthology.orchestra.run.smithi110.stdout:Created osd(s) 1 on host 'smithi110' 2024-09-11T07:54:21.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:21 smithi110 ceph-mon[25392]: pgmap v54: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:54:21.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:21 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:54:21.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:21 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:21.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:21 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:54:21.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:21 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:54:21.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:21 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:21.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:21 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:21.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:21 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:54:21.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:21 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:54:21.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:21 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:21.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:21 smithi110 ceph-mon[25392]: from='osd.1 [v2:172.21.15.110:6810/1928520079,v1:172.21.15.110:6811/1928520079]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T07:54:21.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:21 smithi110 ceph-mon[31709]: pgmap v54: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:54:21.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:21 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:54:21.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:21 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:21.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:21 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:54:21.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:21 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:54:21.720 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:21 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:21.720 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:21 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:21.720 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:21 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:54:21.720 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:21 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:54:21.720 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:21 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:21.720 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:21 smithi110 ceph-mon[31709]: from='osd.1 [v2:172.21.15.110:6810/1928520079,v1:172.21.15.110:6811/1928520079]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T07:54:21.748 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:21 smithi139 ceph-mon[27311]: pgmap v54: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:54:21.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:21 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:54:21.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:21 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:21.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:21 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:54:21.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:21 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:54:21.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:21 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:21.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:21 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:21.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:21 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:54:21.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:21 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:54:21.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:21 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:21.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:21 smithi139 ceph-mon[27311]: from='osd.1 [v2:172.21.15.110:6810/1928520079,v1:172.21.15.110:6811/1928520079]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T07:54:21.765 DEBUG:teuthology.orchestra.run.smithi110:osd.1> sudo journalctl -f -n 0 -u ceph-41bd1096-7012-11ef-bcea-c7b262605968@osd.1.service 2024-09-11T07:54:21.767 INFO:tasks.cephadm:Deploying osd.2 on smithi110 with /dev/nvme3n1... 2024-09-11T07:54:21.768 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/nvme3n1 2024-09-11T07:54:22.045 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:54:22.720 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:22 smithi110 ceph-mon[31709]: Adjusting osd_memory_target on smithi110 to 7520M 2024-09-11T07:54:22.720 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:22 smithi110 ceph-mon[31709]: from='osd.1 [v2:172.21.15.110:6810/1928520079,v1:172.21.15.110:6811/1928520079]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-11T07:54:22.720 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:22 smithi110 ceph-mon[31709]: osdmap e11: 2 total, 1 up, 2 in 2024-09-11T07:54:22.721 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:22 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:54:22.721 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:22 smithi110 ceph-mon[31709]: from='osd.1 [v2:172.21.15.110:6810/1928520079,v1:172.21.15.110:6811/1928520079]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-09-11T07:54:22.721 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:22 smithi110 ceph-mon[25392]: Adjusting osd_memory_target on smithi110 to 7520M 2024-09-11T07:54:22.721 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:22 smithi110 ceph-mon[25392]: from='osd.1 [v2:172.21.15.110:6810/1928520079,v1:172.21.15.110:6811/1928520079]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-11T07:54:22.721 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:22 smithi110 ceph-mon[25392]: osdmap e11: 2 total, 1 up, 2 in 2024-09-11T07:54:22.721 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:22 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:54:22.721 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:22 smithi110 ceph-mon[25392]: from='osd.1 [v2:172.21.15.110:6810/1928520079,v1:172.21.15.110:6811/1928520079]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-09-11T07:54:22.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:22 smithi139 ceph-mon[27311]: Adjusting osd_memory_target on smithi110 to 7520M 2024-09-11T07:54:22.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:22 smithi139 ceph-mon[27311]: from='osd.1 [v2:172.21.15.110:6810/1928520079,v1:172.21.15.110:6811/1928520079]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-11T07:54:22.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:22 smithi139 ceph-mon[27311]: osdmap e11: 2 total, 1 up, 2 in 2024-09-11T07:54:22.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:22 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:54:22.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:22 smithi139 ceph-mon[27311]: from='osd.1 [v2:172.21.15.110:6810/1928520079,v1:172.21.15.110:6811/1928520079]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-09-11T07:54:23.387 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 07:54:23 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-1[39572]: 2024-09-11T07:54:23.177+0000 7fdf49c4d640 -1 osd.1 0 waiting for initial osdmap 2024-09-11T07:54:23.387 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 07:54:23 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-1[39572]: 2024-09-11T07:54:23.180+0000 7fdf44283640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T07:54:23.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:23 smithi110 ceph-mon[25392]: pgmap v55: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:54:23.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:23 smithi110 ceph-mon[25392]: from='osd.1 [v2:172.21.15.110:6810/1928520079,v1:172.21.15.110:6811/1928520079]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-09-11T07:54:23.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:23 smithi110 ceph-mon[25392]: osdmap e12: 2 total, 1 up, 2 in 2024-09-11T07:54:23.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:23 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:54:23.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:23 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:54:23.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:23 smithi110 ceph-mon[31709]: pgmap v55: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:54:23.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:23 smithi110 ceph-mon[31709]: from='osd.1 [v2:172.21.15.110:6810/1928520079,v1:172.21.15.110:6811/1928520079]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-09-11T07:54:23.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:23 smithi110 ceph-mon[31709]: osdmap e12: 2 total, 1 up, 2 in 2024-09-11T07:54:23.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:23 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:54:23.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:23 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:54:23.748 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:23 smithi139 ceph-mon[27311]: pgmap v55: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:54:23.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:23 smithi139 ceph-mon[27311]: from='osd.1 [v2:172.21.15.110:6810/1928520079,v1:172.21.15.110:6811/1928520079]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-09-11T07:54:23.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:23 smithi139 ceph-mon[27311]: osdmap e12: 2 total, 1 up, 2 in 2024-09-11T07:54:23.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:23 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:54:23.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:23 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:54:23.803 INFO:teuthology.orchestra.run.smithi110.stderr:--> Zapping: /dev/nvme3n1 2024-09-11T07:54:23.803 INFO:teuthology.orchestra.run.smithi110.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-09-11T07:54:23.806 INFO:teuthology.orchestra.run.smithi110.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme3n1 bs=1M count=10 conv=fsync 2024-09-11T07:54:23.818 INFO:teuthology.orchestra.run.smithi110.stderr: stderr: 10+0 records in 2024-09-11T07:54:23.819 INFO:teuthology.orchestra.run.smithi110.stderr:10+0 records out 2024-09-11T07:54:23.819 INFO:teuthology.orchestra.run.smithi110.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0105155 s, 997 MB/s 2024-09-11T07:54:23.819 INFO:teuthology.orchestra.run.smithi110.stderr:--> Zapping successful for: 2024-09-11T07:54:24.554 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi110:/dev/nvme3n1 2024-09-11T07:54:24.792 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:54:25.468 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:25 smithi110 ceph-mon[31709]: purged_snaps scrub starts 2024-09-11T07:54:25.468 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:25 smithi110 ceph-mon[31709]: purged_snaps scrub ok 2024-09-11T07:54:25.468 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:25 smithi110 ceph-mon[31709]: pgmap v58: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:54:25.468 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:25 smithi110 ceph-mon[31709]: osd.1 [v2:172.21.15.110:6810/1928520079,v1:172.21.15.110:6811/1928520079] boot 2024-09-11T07:54:25.468 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:25 smithi110 ceph-mon[31709]: osdmap e13: 2 total, 2 up, 2 in 2024-09-11T07:54:25.468 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:25 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:54:25.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:25 smithi110 ceph-mon[25392]: purged_snaps scrub starts 2024-09-11T07:54:25.469 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:25 smithi110 ceph-mon[25392]: purged_snaps scrub ok 2024-09-11T07:54:25.469 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:25 smithi110 ceph-mon[25392]: pgmap v58: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:54:25.469 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:25 smithi110 ceph-mon[25392]: osd.1 [v2:172.21.15.110:6810/1928520079,v1:172.21.15.110:6811/1928520079] boot 2024-09-11T07:54:25.469 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:25 smithi110 ceph-mon[25392]: osdmap e13: 2 total, 2 up, 2 in 2024-09-11T07:54:25.469 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:25 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:54:25.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:25 smithi139 ceph-mon[27311]: purged_snaps scrub starts 2024-09-11T07:54:25.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:25 smithi139 ceph-mon[27311]: purged_snaps scrub ok 2024-09-11T07:54:25.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:25 smithi139 ceph-mon[27311]: pgmap v58: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:54:25.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:25 smithi139 ceph-mon[27311]: osd.1 [v2:172.21.15.110:6810/1928520079,v1:172.21.15.110:6811/1928520079] boot 2024-09-11T07:54:25.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:25 smithi139 ceph-mon[27311]: osdmap e13: 2 total, 2 up, 2 in 2024-09-11T07:54:25.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:25 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:54:26.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:26 smithi110 ceph-mon[25392]: osdmap e14: 2 total, 2 up, 2 in 2024-09-11T07:54:26.468 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:26 smithi110 ceph-mon[31709]: osdmap e14: 2 total, 2 up, 2 in 2024-09-11T07:54:26.500 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:26 smithi139 ceph-mon[27311]: osdmap e14: 2 total, 2 up, 2 in 2024-09-11T07:54:27.455 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:27 smithi110 ceph-mon[31709]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:54:27.455 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:27 smithi110 ceph-mon[31709]: from='client.14265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:/dev/nvme3n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:54:27.455 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:27 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:54:27.455 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:27 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:54:27.456 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:27 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:54:27.456 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:27 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:27.456 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:27 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:27.456 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:27 smithi110 ceph-mon[31709]: Detected new or changed devices on smithi110 2024-09-11T07:54:27.456 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:27 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:54:27.456 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:27 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:27.456 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:27 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:54:27.456 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:27 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:54:27.456 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:27 smithi110 ceph-mon[31709]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:54:27.456 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:27 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:27.457 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:27 smithi110 ceph-mon[25392]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:54:27.457 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:27 smithi110 ceph-mon[25392]: from='client.14265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:/dev/nvme3n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:54:27.457 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:27 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:54:27.457 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:27 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:54:27.457 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:27 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:54:27.457 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:27 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:27.457 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:27 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:27.457 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:27 smithi110 ceph-mon[25392]: Detected new or changed devices on smithi110 2024-09-11T07:54:27.457 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:27 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:54:27.457 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:27 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:27.457 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:27 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:54:27.457 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:27 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:54:27.457 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:27 smithi110 ceph-mon[25392]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:54:27.458 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:27 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:27.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:27 smithi139 ceph-mon[27311]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:54:27.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:27 smithi139 ceph-mon[27311]: from='client.14265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:/dev/nvme3n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:54:27.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:27 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:54:27.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:27 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:54:27.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:27 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:54:27.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:27 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:27.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:27 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:27.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:27 smithi139 ceph-mon[27311]: Detected new or changed devices on smithi110 2024-09-11T07:54:27.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:27 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:54:27.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:27 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:27.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:27 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:54:27.500 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:27 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:54:27.500 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:27 smithi139 ceph-mon[27311]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:54:27.500 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:27 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:28.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:28 smithi110 ceph-mon[25392]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-09-11T07:54:28.468 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:28 smithi110 ceph-mon[31709]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-09-11T07:54:28.498 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:28 smithi139 ceph-mon[27311]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-09-11T07:54:29.443 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:29 smithi110 ceph-mon[25392]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:54:29.444 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:29 smithi110 ceph-mon[31709]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:54:29.498 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:29 smithi139 ceph-mon[27311]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:54:30.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:30 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:30.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:30 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:30.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:30 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:54:30.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:30 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:30.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:30 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:54:30.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:30 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:54:30.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:30 smithi110 ceph-mon[31709]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:54:30.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:30 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:30.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:30 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:30.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:30 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:30.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:30 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:54:30.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:30 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:30.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:30 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:54:30.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:30 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:54:30.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:30 smithi110 ceph-mon[25392]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:54:30.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:30 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:31.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:30 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:31.279 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:30 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:31.279 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:30 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:54:31.279 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:30 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:31.279 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:30 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:54:31.279 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:30 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:54:31.279 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:30 smithi139 ceph-mon[27311]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:54:31.279 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:30 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:32.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:31 smithi110 ceph-mon[25392]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2024-09-11T07:54:32.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:31 smithi110 ceph-mon[25392]: Cluster is now healthy 2024-09-11T07:54:32.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:31 smithi110 ceph-mon[25392]: from='client.? 172.21.15.110:0/606911653' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b413fbb5-f09d-4a04-a72d-1cc5d554f90d"}]: dispatch 2024-09-11T07:54:32.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:31 smithi110 ceph-mon[25392]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b413fbb5-f09d-4a04-a72d-1cc5d554f90d"}]: dispatch 2024-09-11T07:54:32.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:31 smithi110 ceph-mon[25392]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b413fbb5-f09d-4a04-a72d-1cc5d554f90d"}]': finished 2024-09-11T07:54:32.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:31 smithi110 ceph-mon[25392]: osdmap e15: 3 total, 2 up, 3 in 2024-09-11T07:54:32.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:31 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:54:32.219 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:31 smithi110 ceph-mon[31709]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2024-09-11T07:54:32.219 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:31 smithi110 ceph-mon[31709]: Cluster is now healthy 2024-09-11T07:54:32.219 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:31 smithi110 ceph-mon[31709]: from='client.? 172.21.15.110:0/606911653' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b413fbb5-f09d-4a04-a72d-1cc5d554f90d"}]: dispatch 2024-09-11T07:54:32.219 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:31 smithi110 ceph-mon[31709]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b413fbb5-f09d-4a04-a72d-1cc5d554f90d"}]: dispatch 2024-09-11T07:54:32.219 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:31 smithi110 ceph-mon[31709]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b413fbb5-f09d-4a04-a72d-1cc5d554f90d"}]': finished 2024-09-11T07:54:32.219 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:31 smithi110 ceph-mon[31709]: osdmap e15: 3 total, 2 up, 3 in 2024-09-11T07:54:32.219 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:31 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:54:32.248 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:31 smithi139 ceph-mon[27311]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2024-09-11T07:54:32.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:31 smithi139 ceph-mon[27311]: Cluster is now healthy 2024-09-11T07:54:32.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:31 smithi139 ceph-mon[27311]: from='client.? 172.21.15.110:0/606911653' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b413fbb5-f09d-4a04-a72d-1cc5d554f90d"}]: dispatch 2024-09-11T07:54:32.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:31 smithi139 ceph-mon[27311]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b413fbb5-f09d-4a04-a72d-1cc5d554f90d"}]: dispatch 2024-09-11T07:54:32.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:31 smithi139 ceph-mon[27311]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b413fbb5-f09d-4a04-a72d-1cc5d554f90d"}]': finished 2024-09-11T07:54:32.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:31 smithi139 ceph-mon[27311]: osdmap e15: 3 total, 2 up, 3 in 2024-09-11T07:54:32.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:31 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:54:33.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:32 smithi110 ceph-mon[25392]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:54:33.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:32 smithi110 ceph-mon[31709]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:54:33.248 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:32 smithi139 ceph-mon[27311]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:54:34.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:33 smithi110 ceph-mon[31709]: from='client.? 172.21.15.110:0/97607334' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:54:34.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:33 smithi110 ceph-mon[25392]: from='client.? 172.21.15.110:0/97607334' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:54:34.248 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:33 smithi139 ceph-mon[27311]: from='client.? 172.21.15.110:0/97607334' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:54:35.118 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:34 smithi110 ceph-mon[25392]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:54:35.118 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:34 smithi110 ceph-mon[31709]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:54:35.248 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:34 smithi139 ceph-mon[27311]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:54:37.165 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:36 smithi110 ceph-mon[25392]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:54:37.166 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:36 smithi110 ceph-mon[31709]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:54:37.178 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:36 smithi139 ceph-mon[27311]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:54:38.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:38 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:38.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:38 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:38.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:38 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:38.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:38 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-11T07:54:38.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:38 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:54:38.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:38 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:38.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:38 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:38.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:38 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:38.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:38 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-11T07:54:38.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:38 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:54:38.748 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:38 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:38.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:38 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:38.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:38 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:38.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:38 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-11T07:54:38.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:38 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:54:39.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:39 smithi110 ceph-mon[25392]: pgmap v69: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:54:39.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:39 smithi110 ceph-mon[25392]: Deploying daemon osd.2 on smithi110 2024-09-11T07:54:39.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:39 smithi110 ceph-mon[31709]: pgmap v69: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:54:39.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:39 smithi110 ceph-mon[31709]: Deploying daemon osd.2 on smithi110 2024-09-11T07:54:39.748 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:39 smithi139 ceph-mon[27311]: pgmap v69: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:54:39.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:39 smithi139 ceph-mon[27311]: Deploying daemon osd.2 on smithi110 2024-09-11T07:54:41.686 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:41 smithi110 ceph-mon[25392]: pgmap v70: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:54:41.686 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:41 smithi110 ceph-mon[31709]: pgmap v70: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:54:41.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:41 smithi139 ceph-mon[27311]: pgmap v70: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:54:43.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:43 smithi110 ceph-mon[25392]: pgmap v71: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:54:43.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:43 smithi110 ceph-mon[31709]: pgmap v71: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:54:43.748 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:43 smithi139 ceph-mon[27311]: pgmap v71: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:54:44.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:44 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:54:44.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:44 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:44.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:44 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:54:44.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:44 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:54:44.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:44 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:44.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:44 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:44.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:44 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:54:44.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:44 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:44.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:44 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:54:44.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:44 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:54:44.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:44 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:44.720 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:44 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:44.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:44 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:54:44.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:44 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:44.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:44 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:54:44.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:44 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:54:44.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:44 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:44.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:44 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:45.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:45 smithi110 ceph-mon[25392]: pgmap v72: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:54:45.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:45 smithi110 ceph-mon[31709]: pgmap v72: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:54:45.998 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:45 smithi139 ceph-mon[27311]: pgmap v72: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:54:47.485 INFO:teuthology.orchestra.run.smithi110.stdout:Created osd(s) 2 on host 'smithi110' 2024-09-11T07:54:47.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:47 smithi110 ceph-mon[25392]: pgmap v73: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:54:47.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:47 smithi110 ceph-mon[25392]: from='osd.2 [v2:172.21.15.110:6818/3894315382,v1:172.21.15.110:6819/3894315382]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T07:54:47.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:47 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:54:47.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:47 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:47.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:47 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:54:47.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:47 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:54:47.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:47 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:47.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:47 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:54:47.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:47 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:47.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:47 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:54:47.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:47 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:54:47.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:47 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:47.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:47 smithi110 ceph-mon[31709]: pgmap v73: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:54:47.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:47 smithi110 ceph-mon[31709]: from='osd.2 [v2:172.21.15.110:6818/3894315382,v1:172.21.15.110:6819/3894315382]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T07:54:47.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:47 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:54:47.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:47 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:47.720 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:47 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:54:47.720 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:47 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:54:47.720 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:47 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:47.720 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:47 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:54:47.720 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:47 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:47.720 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:47 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:54:47.720 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:47 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:54:47.720 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:47 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:47.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:47 smithi139 ceph-mon[27311]: pgmap v73: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:54:47.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:47 smithi139 ceph-mon[27311]: from='osd.2 [v2:172.21.15.110:6818/3894315382,v1:172.21.15.110:6819/3894315382]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T07:54:47.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:47 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:54:47.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:47 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:47.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:47 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:54:47.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:47 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:54:47.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:47 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:48.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:47 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:54:48.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:47 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:48.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:47 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:54:48.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:47 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:54:48.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:47 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:48.085 DEBUG:teuthology.orchestra.run.smithi110:osd.2> sudo journalctl -f -n 0 -u ceph-41bd1096-7012-11ef-bcea-c7b262605968@osd.2.service 2024-09-11T07:54:48.088 INFO:tasks.cephadm:Deploying osd.3 on smithi110 with /dev/nvme4n1... 2024-09-11T07:54:48.088 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/nvme4n1 2024-09-11T07:54:48.597 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:54:48.868 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:48 smithi110 ceph-mon[25392]: Adjusting osd_memory_target on smithi110 to 5013M 2024-09-11T07:54:48.869 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:48 smithi110 ceph-mon[25392]: from='osd.2 [v2:172.21.15.110:6818/3894315382,v1:172.21.15.110:6819/3894315382]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-11T07:54:48.869 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:48 smithi110 ceph-mon[25392]: osdmap e16: 3 total, 2 up, 3 in 2024-09-11T07:54:48.869 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:48 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:54:48.869 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:48 smithi110 ceph-mon[25392]: from='osd.2 [v2:172.21.15.110:6818/3894315382,v1:172.21.15.110:6819/3894315382]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-09-11T07:54:48.869 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:48 smithi110 ceph-mon[31709]: Adjusting osd_memory_target on smithi110 to 5013M 2024-09-11T07:54:48.869 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:48 smithi110 ceph-mon[31709]: from='osd.2 [v2:172.21.15.110:6818/3894315382,v1:172.21.15.110:6819/3894315382]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-11T07:54:48.869 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:48 smithi110 ceph-mon[31709]: osdmap e16: 3 total, 2 up, 3 in 2024-09-11T07:54:48.869 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:48 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:54:48.870 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:48 smithi110 ceph-mon[31709]: from='osd.2 [v2:172.21.15.110:6818/3894315382,v1:172.21.15.110:6819/3894315382]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-09-11T07:54:48.870 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 11 07:54:48 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-2[43117]: 2024-09-11T07:54:48.619+0000 7f459d05d640 -1 osd.2 0 waiting for initial osdmap 2024-09-11T07:54:48.870 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 11 07:54:48 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-2[43117]: 2024-09-11T07:54:48.623+0000 7f4598e96640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T07:54:48.998 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:48 smithi139 ceph-mon[27311]: Adjusting osd_memory_target on smithi110 to 5013M 2024-09-11T07:54:48.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:48 smithi139 ceph-mon[27311]: from='osd.2 [v2:172.21.15.110:6818/3894315382,v1:172.21.15.110:6819/3894315382]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-11T07:54:48.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:48 smithi139 ceph-mon[27311]: osdmap e16: 3 total, 2 up, 3 in 2024-09-11T07:54:48.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:48 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:54:48.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:48 smithi139 ceph-mon[27311]: from='osd.2 [v2:172.21.15.110:6818/3894315382,v1:172.21.15.110:6819/3894315382]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-09-11T07:54:49.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:49 smithi110 ceph-mon[25392]: pgmap v75: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:54:49.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:49 smithi110 ceph-mon[25392]: from='osd.2 [v2:172.21.15.110:6818/3894315382,v1:172.21.15.110:6819/3894315382]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-09-11T07:54:49.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:49 smithi110 ceph-mon[25392]: osdmap e17: 3 total, 2 up, 3 in 2024-09-11T07:54:49.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:49 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:54:49.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:49 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:54:49.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:49 smithi110 ceph-mon[31709]: pgmap v75: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:54:49.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:49 smithi110 ceph-mon[31709]: from='osd.2 [v2:172.21.15.110:6818/3894315382,v1:172.21.15.110:6819/3894315382]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-09-11T07:54:49.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:49 smithi110 ceph-mon[31709]: osdmap e17: 3 total, 2 up, 3 in 2024-09-11T07:54:49.970 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:49 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:54:49.970 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:49 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:54:49.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:49 smithi139 ceph-mon[27311]: pgmap v75: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:54:49.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:49 smithi139 ceph-mon[27311]: from='osd.2 [v2:172.21.15.110:6818/3894315382,v1:172.21.15.110:6819/3894315382]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-09-11T07:54:49.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:49 smithi139 ceph-mon[27311]: osdmap e17: 3 total, 2 up, 3 in 2024-09-11T07:54:49.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:49 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:54:49.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:49 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:54:50.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:50 smithi110 ceph-mon[25392]: purged_snaps scrub starts 2024-09-11T07:54:50.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:50 smithi110 ceph-mon[25392]: purged_snaps scrub ok 2024-09-11T07:54:50.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:50 smithi110 ceph-mon[25392]: osd.2 [v2:172.21.15.110:6818/3894315382,v1:172.21.15.110:6819/3894315382] boot 2024-09-11T07:54:50.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:50 smithi110 ceph-mon[25392]: osdmap e18: 3 total, 3 up, 3 in 2024-09-11T07:54:50.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:50 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:54:50.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:50 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:50.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:50 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:50.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:50 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:54:50.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:50 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:50.970 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:50 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:54:50.970 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:50 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:54:50.970 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:50 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:50.970 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:50 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-09-11T07:54:50.970 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:50 smithi110 ceph-mon[31709]: purged_snaps scrub starts 2024-09-11T07:54:50.970 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:50 smithi110 ceph-mon[31709]: purged_snaps scrub ok 2024-09-11T07:54:50.970 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:50 smithi110 ceph-mon[31709]: osd.2 [v2:172.21.15.110:6818/3894315382,v1:172.21.15.110:6819/3894315382] boot 2024-09-11T07:54:50.970 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:50 smithi110 ceph-mon[31709]: osdmap e18: 3 total, 3 up, 3 in 2024-09-11T07:54:50.970 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:50 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:54:50.970 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:50 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:50.971 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:50 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:50.971 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:50 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:54:50.971 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:50 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:50.971 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:50 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:54:50.971 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:50 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:54:50.971 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:50 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:50.971 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:50 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-09-11T07:54:50.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:50 smithi139 ceph-mon[27311]: purged_snaps scrub starts 2024-09-11T07:54:50.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:50 smithi139 ceph-mon[27311]: purged_snaps scrub ok 2024-09-11T07:54:50.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:50 smithi139 ceph-mon[27311]: osd.2 [v2:172.21.15.110:6818/3894315382,v1:172.21.15.110:6819/3894315382] boot 2024-09-11T07:54:50.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:50 smithi139 ceph-mon[27311]: osdmap e18: 3 total, 3 up, 3 in 2024-09-11T07:54:50.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:50 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:54:50.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:50 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:50.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:50 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:51.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:50 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:54:51.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:50 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:51.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:50 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:54:51.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:50 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:54:51.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:50 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:51.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:50 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-09-11T07:54:51.098 INFO:teuthology.orchestra.run.smithi110.stderr:--> Zapping: /dev/nvme4n1 2024-09-11T07:54:51.098 INFO:teuthology.orchestra.run.smithi110.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-09-11T07:54:51.102 INFO:teuthology.orchestra.run.smithi110.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme4n1 bs=1M count=10 conv=fsync 2024-09-11T07:54:51.114 INFO:teuthology.orchestra.run.smithi110.stderr: stderr: 10+0 records in 2024-09-11T07:54:51.114 INFO:teuthology.orchestra.run.smithi110.stderr:10+0 records out 2024-09-11T07:54:51.114 INFO:teuthology.orchestra.run.smithi110.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0102401 s, 1.0 GB/s 2024-09-11T07:54:51.114 INFO:teuthology.orchestra.run.smithi110.stderr:--> Zapping successful for: 2024-09-11T07:54:51.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:51 smithi110 ceph-mon[25392]: pgmap v78: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:54:51.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:51 smithi110 ceph-mon[25392]: pgmap v79: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:54:51.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:51 smithi110 ceph-mon[25392]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-09-11T07:54:51.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:51 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-09-11T07:54:51.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:51 smithi110 ceph-mon[25392]: osdmap e19: 3 total, 3 up, 3 in 2024-09-11T07:54:51.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:51 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T07:54:51.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:51 smithi110 ceph-mon[31709]: pgmap v78: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:54:51.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:51 smithi110 ceph-mon[31709]: pgmap v79: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:54:51.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:51 smithi110 ceph-mon[31709]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-09-11T07:54:51.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:51 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-09-11T07:54:51.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:51 smithi110 ceph-mon[31709]: osdmap e19: 3 total, 3 up, 3 in 2024-09-11T07:54:51.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:51 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T07:54:51.759 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi110:/dev/nvme4n1 2024-09-11T07:54:51.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:51 smithi139 ceph-mon[27311]: pgmap v78: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:54:51.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:51 smithi139 ceph-mon[27311]: pgmap v79: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:54:51.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:51 smithi139 ceph-mon[27311]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-09-11T07:54:51.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:51 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-09-11T07:54:51.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:51 smithi139 ceph-mon[27311]: osdmap e19: 3 total, 3 up, 3 in 2024-09-11T07:54:51.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:51 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T07:54:52.020 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 11 07:54:51 smithi110 sudo[44462]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme1n1 2024-09-11T07:54:52.020 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 11 07:54:51 smithi110 sudo[44462]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T07:54:52.020 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 11 07:54:51 smithi110 sudo[44462]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T07:54:52.020 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 11 07:54:51 smithi110 sudo[44462]: pam_unix(sudo:session): session closed for user root 2024-09-11T07:54:52.249 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:54:52.371 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 11 07:54:52 smithi110 sudo[44481]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme linux smart-log-add --json /dev/nvme1n1 2024-09-11T07:54:52.371 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 11 07:54:52 smithi110 sudo[44481]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T07:54:52.371 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 11 07:54:52 smithi110 sudo[44481]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T07:54:52.372 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 11 07:54:52 smithi110 sudo[44481]: pam_unix(sudo:session): session closed for user root 2024-09-11T07:54:52.636 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 07:54:52 smithi110 sudo[44554]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme2n1 2024-09-11T07:54:52.636 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 07:54:52 smithi110 sudo[44554]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T07:54:52.636 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 07:54:52 smithi110 sudo[44554]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T07:54:52.636 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 07:54:52 smithi110 sudo[44554]: pam_unix(sudo:session): session closed for user root 2024-09-11T07:54:52.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:52 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-11T07:54:52.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:52 smithi110 ceph-mon[31709]: osdmap e20: 3 total, 3 up, 3 in 2024-09-11T07:54:52.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:52 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:52.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:52 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-11T07:54:52.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:52 smithi110 ceph-mon[25392]: osdmap e20: 3 total, 3 up, 3 in 2024-09-11T07:54:52.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:52 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:52.969 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 07:54:52 smithi110 sudo[44574]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme linux smart-log-add --json /dev/nvme2n1 2024-09-11T07:54:52.969 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 07:54:52 smithi110 sudo[44574]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T07:54:52.969 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 07:54:52 smithi110 sudo[44574]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T07:54:52.969 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 07:54:52 smithi110 sudo[44574]: pam_unix(sudo:session): session closed for user root 2024-09-11T07:54:52.998 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:52 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-11T07:54:52.998 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:52 smithi139 ceph-mon[27311]: osdmap e20: 3 total, 3 up, 3 in 2024-09-11T07:54:52.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:52 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:53.283 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 11 07:54:53 smithi110 sudo[44592]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme3n1 2024-09-11T07:54:53.283 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 11 07:54:53 smithi110 sudo[44592]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T07:54:53.283 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 11 07:54:53 smithi110 sudo[44592]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T07:54:53.283 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 11 07:54:53 smithi110 sudo[44592]: pam_unix(sudo:session): session closed for user root 2024-09-11T07:54:53.561 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 11 07:54:53 smithi110 sudo[44597]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme linux smart-log-add --json /dev/nvme3n1 2024-09-11T07:54:53.561 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 11 07:54:53 smithi110 sudo[44597]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T07:54:53.561 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 11 07:54:53 smithi110 sudo[44597]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T07:54:53.561 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 11 07:54:53 smithi110 sudo[44597]: pam_unix(sudo:session): session closed for user root 2024-09-11T07:54:53.826 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:53 smithi110 sudo[44653]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T07:54:53.826 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:53 smithi110 sudo[44653]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T07:54:53.826 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:53 smithi110 sudo[44653]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T07:54:53.826 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:53 smithi110 sudo[44653]: pam_unix(sudo:session): session closed for user root 2024-09-11T07:54:54.093 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:53 smithi139 ceph-mon[27311]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:54:54.094 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:53 smithi139 ceph-mon[27311]: osdmap e21: 3 total, 3 up, 3 in 2024-09-11T07:54:54.094 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:53 smithi139 ceph-mon[27311]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-11T07:54:54.094 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:53 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:54.094 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:53 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:54.094 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:53 smithi139 ceph-mon[27311]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T07:54:54.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:53 smithi110 ceph-mon[31709]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:54:54.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:53 smithi110 ceph-mon[31709]: osdmap e21: 3 total, 3 up, 3 in 2024-09-11T07:54:54.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:53 smithi110 ceph-mon[31709]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-11T07:54:54.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:53 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:54.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:53 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:54.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:53 smithi110 ceph-mon[31709]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T07:54:54.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:53 smithi110 sudo[44683]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T07:54:54.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:53 smithi110 sudo[44683]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T07:54:54.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:53 smithi110 sudo[44683]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T07:54:54.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:53 smithi110 sudo[44683]: pam_unix(sudo:session): session closed for user root 2024-09-11T07:54:54.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:53 smithi110 ceph-mon[25392]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:54:54.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:53 smithi110 ceph-mon[25392]: osdmap e21: 3 total, 3 up, 3 in 2024-09-11T07:54:54.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:53 smithi110 ceph-mon[25392]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-11T07:54:54.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:53 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:54.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:53 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:54.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:53 smithi110 ceph-mon[25392]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T07:54:54.347 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:54 smithi139 sudo[30897]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T07:54:54.347 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:54 smithi139 sudo[30897]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T07:54:54.347 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:54 smithi139 sudo[30897]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T07:54:54.347 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:54 smithi139 sudo[30897]: pam_unix(sudo:session): session closed for user root 2024-09-11T07:54:54.748 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:54 smithi139 sudo[30900]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T07:54:54.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:54 smithi139 sudo[30900]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T07:54:54.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:54 smithi139 sudo[30900]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T07:54:54.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:54 smithi139 sudo[30900]: pam_unix(sudo:session): session closed for user root 2024-09-11T07:54:54.880 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:54 smithi110 sudo[44879]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T07:54:54.881 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:54 smithi110 sudo[44879]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T07:54:54.881 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:54 smithi110 sudo[44879]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T07:54:54.881 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:54 smithi110 sudo[44879]: pam_unix(sudo:session): session closed for user root 2024-09-11T07:54:55.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:54 smithi110 ceph-mon[25392]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T07:54:55.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:54 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:54:55.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:54 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:54:55.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:54 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:54:55.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:54 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:55.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:54 smithi110 ceph-mon[25392]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T07:54:55.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:54 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:54:55.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:54 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:54:55.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:54 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:54:55.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:54 smithi110 ceph-mon[25392]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:54:55.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:54 smithi110 ceph-mon[25392]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:54:55.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:54 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:54:55.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:54 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:54:55.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:54 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:54:55.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:54 smithi110 ceph-mon[25392]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T07:54:55.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:54 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:55.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:54 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:55.220 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:54 smithi110 ceph-mon[25392]: Detected new or changed devices on smithi110 2024-09-11T07:54:55.220 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:54 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:54:55.220 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:54 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:55.220 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:54 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:54:55.220 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:54 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:54:55.220 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:54 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:55.220 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:54 smithi110 ceph-mon[25392]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:54:55.220 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:54 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:54:55.220 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:54 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:54:55.220 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:54 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:54:55.221 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:54 smithi110 sudo[44895]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T07:54:55.221 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:54 smithi110 sudo[44895]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T07:54:55.221 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:54 smithi110 sudo[44895]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T07:54:55.221 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:54 smithi110 sudo[44895]: pam_unix(sudo:session): session closed for user root 2024-09-11T07:54:55.221 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:54 smithi110 ceph-mon[31709]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T07:54:55.221 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:54 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:54:55.221 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:54 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:54:55.221 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:54 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:54:55.221 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:54 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:55.221 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:54 smithi110 ceph-mon[31709]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T07:54:55.221 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:54 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:54:55.221 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:54 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:54:55.222 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:54 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:54:55.222 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:54 smithi110 ceph-mon[31709]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:54:55.222 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:54 smithi110 ceph-mon[31709]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:54:55.222 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:54 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:54:55.222 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:54 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:54:55.222 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:54 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:54:55.222 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:54 smithi110 ceph-mon[31709]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T07:54:55.222 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:54 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:55.222 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:54 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:55.222 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:54 smithi110 ceph-mon[31709]: Detected new or changed devices on smithi110 2024-09-11T07:54:55.222 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:54 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:54:55.222 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:54 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:55.222 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:54 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:54:55.222 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:54 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:54:55.223 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:54 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:55.223 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:54 smithi110 ceph-mon[31709]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:54:55.223 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:54 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:54:55.223 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:54 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:54:55.223 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:54 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:54:55.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:54 smithi139 ceph-mon[27311]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T07:54:55.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:54 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:54:55.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:54 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:54:55.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:54 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:54:55.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:54 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:55.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:54 smithi139 ceph-mon[27311]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T07:54:55.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:54 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:54:55.250 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:54 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:54:55.250 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:54 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:54:55.250 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:54 smithi139 ceph-mon[27311]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:54:55.250 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:54 smithi139 ceph-mon[27311]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:54:55.250 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:54 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:54:55.250 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:54 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:54:55.250 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:54 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:54:55.250 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:54 smithi139 ceph-mon[27311]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T07:54:55.250 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:54 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:55.250 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:54 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:55.250 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:54 smithi139 ceph-mon[27311]: Detected new or changed devices on smithi110 2024-09-11T07:54:55.250 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:54 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:54:55.250 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:54 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:55.250 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:54 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:54:55.251 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:54 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:54:55.251 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:54 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:55.251 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:54 smithi139 ceph-mon[27311]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:54:55.251 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:54 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:54:55.251 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:54 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:54:55.251 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:54 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:54:56.187 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:55 smithi110 ceph-mon[25392]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T07:54:56.187 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:55 smithi110 ceph-mon[25392]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2024-09-11T07:54:56.188 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:55 smithi110 ceph-mon[25392]: Cluster is now healthy 2024-09-11T07:54:56.188 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:55 smithi110 ceph-mon[25392]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T07:54:56.188 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:55 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:56.188 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:55 smithi110 ceph-mon[31709]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T07:54:56.188 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:55 smithi110 ceph-mon[31709]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2024-09-11T07:54:56.188 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:55 smithi110 ceph-mon[31709]: Cluster is now healthy 2024-09-11T07:54:56.188 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:55 smithi110 ceph-mon[31709]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T07:54:56.188 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:55 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:56.248 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:55 smithi139 ceph-mon[27311]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T07:54:56.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:55 smithi139 ceph-mon[27311]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2024-09-11T07:54:56.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:55 smithi139 ceph-mon[27311]: Cluster is now healthy 2024-09-11T07:54:56.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:55 smithi139 ceph-mon[27311]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T07:54:56.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:55 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:57.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:56 smithi110 ceph-mon[25392]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 875 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:54:57.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:56 smithi110 ceph-mon[25392]: from='client.? 172.21.15.110:0/3971018682' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "293d5b4d-11ef-4d6a-9b65-5a6b47472c36"}]: dispatch 2024-09-11T07:54:57.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:56 smithi110 ceph-mon[25392]: from='client.? 172.21.15.110:0/3971018682' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "293d5b4d-11ef-4d6a-9b65-5a6b47472c36"}]': finished 2024-09-11T07:54:57.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:56 smithi110 ceph-mon[25392]: osdmap e22: 4 total, 3 up, 4 in 2024-09-11T07:54:57.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:56 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:54:57.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:56 smithi110 ceph-mon[31709]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 875 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:54:57.219 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:56 smithi110 ceph-mon[31709]: from='client.? 172.21.15.110:0/3971018682' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "293d5b4d-11ef-4d6a-9b65-5a6b47472c36"}]: dispatch 2024-09-11T07:54:57.219 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:56 smithi110 ceph-mon[31709]: from='client.? 172.21.15.110:0/3971018682' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "293d5b4d-11ef-4d6a-9b65-5a6b47472c36"}]': finished 2024-09-11T07:54:57.219 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:56 smithi110 ceph-mon[31709]: osdmap e22: 4 total, 3 up, 4 in 2024-09-11T07:54:57.219 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:56 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:54:57.248 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:56 smithi139 ceph-mon[27311]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 875 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:54:57.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:56 smithi139 ceph-mon[27311]: from='client.? 172.21.15.110:0/3971018682' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "293d5b4d-11ef-4d6a-9b65-5a6b47472c36"}]: dispatch 2024-09-11T07:54:57.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:56 smithi139 ceph-mon[27311]: from='client.? 172.21.15.110:0/3971018682' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "293d5b4d-11ef-4d6a-9b65-5a6b47472c36"}]': finished 2024-09-11T07:54:57.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:56 smithi139 ceph-mon[27311]: osdmap e22: 4 total, 3 up, 4 in 2024-09-11T07:54:57.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:56 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:54:58.963 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:58 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:58.963 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:58 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:58.963 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:58 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:58.963 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:58 smithi139 ceph-mon[27311]: from='client.? 172.21.15.110:0/107758253' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:54:58.967 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:58 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:58.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:58 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:58.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:58 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:58.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:58 smithi110 ceph-mon[25392]: from='client.? 172.21.15.110:0/107758253' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:54:58.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:58 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:58.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:58 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:58.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:58 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:54:58.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:58 smithi110 ceph-mon[31709]: from='client.? 172.21.15.110:0/107758253' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:54:59.748 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:54:59 smithi139 ceph-mon[27311]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 875 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:54:59.859 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:54:59 smithi110 ceph-mon[31709]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 875 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:54:59.859 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:54:59 smithi110 ceph-mon[25392]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 875 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:55:01.932 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:01 smithi110 ceph-mon[31709]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-09-11T07:55:01.933 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:01 smithi110 ceph-mon[25392]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-09-11T07:55:01.998 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:01 smithi139 ceph-mon[27311]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-09-11T07:55:03.876 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:03 smithi110 ceph-mon[25392]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-09-11T07:55:03.876 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:03 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-11T07:55:03.876 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:03 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:55:03.876 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:03 smithi110 ceph-mon[31709]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-09-11T07:55:03.876 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:03 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-11T07:55:03.876 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:03 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:55:03.998 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:03 smithi139 ceph-mon[27311]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-09-11T07:55:03.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:03 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-11T07:55:03.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:03 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:55:04.805 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:04 smithi110 ceph-mon[31709]: Deploying daemon osd.3 on smithi110 2024-09-11T07:55:04.805 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:04 smithi110 ceph-mon[25392]: Deploying daemon osd.3 on smithi110 2024-09-11T07:55:04.998 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:04 smithi139 ceph-mon[27311]: Deploying daemon osd.3 on smithi110 2024-09-11T07:55:05.830 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:05 smithi110 ceph-mon[31709]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-09-11T07:55:05.830 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:05 smithi110 ceph-mon[25392]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-09-11T07:55:05.998 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:05 smithi139 ceph-mon[27311]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-09-11T07:55:07.808 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:07 smithi110 ceph-mon[25392]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-09-11T07:55:07.809 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:07 smithi110 ceph-mon[31709]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-09-11T07:55:07.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:07 smithi139 ceph-mon[27311]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-09-11T07:55:09.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:09 smithi139 ceph-mon[27311]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 66 KiB/s wr, 4 op/s 2024-09-11T07:55:09.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:09 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:55:09.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:09 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:09.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:09 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:55:09.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:09 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:55:09.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:09 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:09.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:09 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:09.919 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:09 smithi110 ceph-mon[31709]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 66 KiB/s wr, 4 op/s 2024-09-11T07:55:09.919 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:09 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:55:09.919 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:09 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:09.919 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:09 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:55:09.919 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:09 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:55:09.919 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:09 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:09.919 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:09 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:09.919 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:09 smithi110 ceph-mon[25392]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 66 KiB/s wr, 4 op/s 2024-09-11T07:55:09.919 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:09 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:55:09.920 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:09 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:09.920 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:09 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:55:09.920 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:09 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:55:09.920 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:09 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:09.920 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:09 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:11.786 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:11 smithi110 ceph-mon[31709]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-11T07:55:11.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:11 smithi110 ceph-mon[25392]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-11T07:55:11.998 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:11 smithi139 ceph-mon[27311]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-11T07:55:12.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:12 smithi110 ceph-mon[25392]: from='osd.3 [v2:172.21.15.110:6826/1640229305,v1:172.21.15.110:6827/1640229305]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-11T07:55:12.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:12 smithi110 ceph-mon[25392]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-11T07:55:12.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:12 smithi110 ceph-mon[31709]: from='osd.3 [v2:172.21.15.110:6826/1640229305,v1:172.21.15.110:6827/1640229305]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-11T07:55:12.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:12 smithi110 ceph-mon[31709]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-11T07:55:12.998 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:12 smithi139 ceph-mon[27311]: from='osd.3 [v2:172.21.15.110:6826/1640229305,v1:172.21.15.110:6827/1640229305]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-11T07:55:12.998 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:12 smithi139 ceph-mon[27311]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-11T07:55:13.299 INFO:teuthology.orchestra.run.smithi110.stdout:Created osd(s) 3 on host 'smithi110' 2024-09-11T07:55:13.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:13 smithi139 ceph-mon[27311]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:55:13.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:13 smithi139 ceph-mon[27311]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-11T07:55:13.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:13 smithi139 ceph-mon[27311]: osdmap e23: 4 total, 3 up, 4 in 2024-09-11T07:55:13.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:13 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:55:13.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:13 smithi139 ceph-mon[27311]: from='osd.3 [v2:172.21.15.110:6826/1640229305,v1:172.21.15.110:6827/1640229305]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-09-11T07:55:13.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:13 smithi139 ceph-mon[27311]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-09-11T07:55:13.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:13 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:55:13.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:13 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:13.750 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:13 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:55:13.750 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:13 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:55:13.750 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:13 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:13.750 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:13 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:55:13.750 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:13 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:55:13.750 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:13 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:13.750 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:13 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:55:13.750 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:13 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:55:13.750 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:13 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:13.887 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:13 smithi110 ceph-mon[31709]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:55:13.887 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:13 smithi110 ceph-mon[31709]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-11T07:55:13.887 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:13 smithi110 ceph-mon[31709]: osdmap e23: 4 total, 3 up, 4 in 2024-09-11T07:55:13.887 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:13 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:55:13.887 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:13 smithi110 ceph-mon[31709]: from='osd.3 [v2:172.21.15.110:6826/1640229305,v1:172.21.15.110:6827/1640229305]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-09-11T07:55:13.887 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:13 smithi110 ceph-mon[31709]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-09-11T07:55:13.887 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:13 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:55:13.887 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:13 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:13.887 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:13 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:55:13.887 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:13 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:55:13.887 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:13 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:13.888 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:13 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:55:13.888 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:13 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:55:13.888 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:13 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:13.888 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:13 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:55:13.888 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:13 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:55:13.888 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:13 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:13.888 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:13 smithi110 ceph-mon[25392]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:55:13.888 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:13 smithi110 ceph-mon[25392]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-11T07:55:13.888 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:13 smithi110 ceph-mon[25392]: osdmap e23: 4 total, 3 up, 4 in 2024-09-11T07:55:13.888 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:13 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:55:13.888 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:13 smithi110 ceph-mon[25392]: from='osd.3 [v2:172.21.15.110:6826/1640229305,v1:172.21.15.110:6827/1640229305]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-09-11T07:55:13.889 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:13 smithi110 ceph-mon[25392]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-09-11T07:55:13.889 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:13 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:55:13.889 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:13 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:13.889 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:13 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:55:13.889 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:13 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:55:13.889 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:13 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:13.889 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:13 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:55:13.889 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:13 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:55:13.889 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:13 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:13.889 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:13 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:55:13.889 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:13 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:55:13.889 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:13 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:13.910 DEBUG:teuthology.orchestra.run.smithi110:osd.3> sudo journalctl -f -n 0 -u ceph-41bd1096-7012-11ef-bcea-c7b262605968@osd.3.service 2024-09-11T07:55:13.912 INFO:tasks.cephadm:Deploying osd.4 on smithi139 with /dev/nvme1n1... 2024-09-11T07:55:13.912 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/nvme1n1 2024-09-11T07:55:14.155 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:55:14.748 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:14 smithi139 ceph-mon[27311]: Adjusting osd_memory_target on smithi110 to 3760M 2024-09-11T07:55:14.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:14 smithi139 ceph-mon[27311]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-09-11T07:55:14.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:14 smithi139 ceph-mon[27311]: osdmap e24: 4 total, 3 up, 4 in 2024-09-11T07:55:14.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:14 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:55:14.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:14 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:55:14.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:14 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:14.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:14 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:14.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:14 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:14.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:14 smithi110 ceph-mon[25392]: Adjusting osd_memory_target on smithi110 to 3760M 2024-09-11T07:55:14.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:14 smithi110 ceph-mon[25392]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-09-11T07:55:14.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:14 smithi110 ceph-mon[25392]: osdmap e24: 4 total, 3 up, 4 in 2024-09-11T07:55:14.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:14 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:55:14.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:14 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:55:14.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:14 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:14.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:14 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:14.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:14 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:14.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:14 smithi110 ceph-mon[31709]: Adjusting osd_memory_target on smithi110 to 3760M 2024-09-11T07:55:14.970 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:14 smithi110 ceph-mon[31709]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-09-11T07:55:14.970 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:14 smithi110 ceph-mon[31709]: osdmap e24: 4 total, 3 up, 4 in 2024-09-11T07:55:14.970 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:14 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:55:14.970 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:14 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:55:14.970 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:14 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:14.970 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:14 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:14.970 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:14 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:15.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:15 smithi139 ceph-mon[27311]: purged_snaps scrub starts 2024-09-11T07:55:15.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:15 smithi139 ceph-mon[27311]: purged_snaps scrub ok 2024-09-11T07:55:15.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:15 smithi139 ceph-mon[27311]: Detected new or changed devices on smithi110 2024-09-11T07:55:15.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:15 smithi139 ceph-mon[27311]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:55:15.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:15 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:55:15.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:15 smithi139 ceph-mon[27311]: osd.3 [v2:172.21.15.110:6826/1640229305,v1:172.21.15.110:6827/1640229305] boot 2024-09-11T07:55:15.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:15 smithi139 ceph-mon[27311]: osdmap e25: 4 total, 4 up, 4 in 2024-09-11T07:55:15.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:15 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:55:15.877 INFO:teuthology.orchestra.run.smithi139.stderr:--> Zapping: /dev/nvme1n1 2024-09-11T07:55:15.877 INFO:teuthology.orchestra.run.smithi139.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-09-11T07:55:15.882 INFO:teuthology.orchestra.run.smithi139.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme1n1 bs=1M count=10 conv=fsync 2024-09-11T07:55:15.896 INFO:teuthology.orchestra.run.smithi139.stderr: stderr: 10+0 records in 2024-09-11T07:55:15.896 INFO:teuthology.orchestra.run.smithi139.stderr:10+0 records out 2024-09-11T07:55:15.896 INFO:teuthology.orchestra.run.smithi139.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0124829 s, 840 MB/s 2024-09-11T07:55:15.896 INFO:teuthology.orchestra.run.smithi139.stderr:--> Zapping successful for: 2024-09-11T07:55:15.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:15 smithi110 ceph-mon[25392]: purged_snaps scrub starts 2024-09-11T07:55:15.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:15 smithi110 ceph-mon[25392]: purged_snaps scrub ok 2024-09-11T07:55:15.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:15 smithi110 ceph-mon[25392]: Detected new or changed devices on smithi110 2024-09-11T07:55:15.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:15 smithi110 ceph-mon[25392]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:55:15.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:15 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:55:15.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:15 smithi110 ceph-mon[25392]: osd.3 [v2:172.21.15.110:6826/1640229305,v1:172.21.15.110:6827/1640229305] boot 2024-09-11T07:55:15.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:15 smithi110 ceph-mon[25392]: osdmap e25: 4 total, 4 up, 4 in 2024-09-11T07:55:15.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:15 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:55:15.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:15 smithi110 ceph-mon[31709]: purged_snaps scrub starts 2024-09-11T07:55:15.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:15 smithi110 ceph-mon[31709]: purged_snaps scrub ok 2024-09-11T07:55:15.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:15 smithi110 ceph-mon[31709]: Detected new or changed devices on smithi110 2024-09-11T07:55:15.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:15 smithi110 ceph-mon[31709]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:55:15.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:15 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:55:15.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:15 smithi110 ceph-mon[31709]: osd.3 [v2:172.21.15.110:6826/1640229305,v1:172.21.15.110:6827/1640229305] boot 2024-09-11T07:55:15.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:15 smithi110 ceph-mon[31709]: osdmap e25: 4 total, 4 up, 4 in 2024-09-11T07:55:15.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:15 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:55:16.487 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi139:/dev/nvme1n1 2024-09-11T07:55:16.729 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:55:16.850 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:16 smithi139 ceph-mon[27311]: osdmap e26: 4 total, 4 up, 4 in 2024-09-11T07:55:16.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:16 smithi110 ceph-mon[25392]: osdmap e26: 4 total, 4 up, 4 in 2024-09-11T07:55:16.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:16 smithi110 ceph-mon[31709]: osdmap e26: 4 total, 4 up, 4 in 2024-09-11T07:55:17.740 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:17 smithi139 ceph-mon[27311]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:55:17.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:17 smithi110 ceph-mon[25392]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:55:17.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:17 smithi110 ceph-mon[31709]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:55:18.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:18 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:18.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:18 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:18.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:18 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:19.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:18 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:19.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:18 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:19.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:18 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:19.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:18 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:19.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:18 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:19.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:18 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:19.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:19 smithi139 ceph-mon[27311]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:55:19.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:19 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:55:19.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:19 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:55:19.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:19 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:55:20.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:19 smithi110 ceph-mon[25392]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:55:20.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:19 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:55:20.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:19 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:55:20.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:19 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:55:20.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:19 smithi110 ceph-mon[31709]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:55:20.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:19 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:55:20.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:19 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:55:20.219 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:19 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:55:20.958 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:20 smithi139 ceph-mon[27311]: from='client.24178 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:55:21.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:20 smithi110 ceph-mon[25392]: from='client.24178 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:55:21.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:20 smithi110 ceph-mon[31709]: from='client.24178 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:55:21.932 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:21 smithi139 ceph-mon[27311]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:55:22.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:21 smithi110 ceph-mon[25392]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:55:22.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:21 smithi110 ceph-mon[31709]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:55:22.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:22 smithi139 ceph-mon[27311]: from='client.? 172.21.15.139:0/3248681975' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f9119e96-9863-45b7-b2d5-89bc3eeb1fed"}]: dispatch 2024-09-11T07:55:22.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:22 smithi139 ceph-mon[27311]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f9119e96-9863-45b7-b2d5-89bc3eeb1fed"}]: dispatch 2024-09-11T07:55:22.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:22 smithi139 ceph-mon[27311]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f9119e96-9863-45b7-b2d5-89bc3eeb1fed"}]': finished 2024-09-11T07:55:22.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:22 smithi139 ceph-mon[27311]: osdmap e27: 5 total, 4 up, 5 in 2024-09-11T07:55:22.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:22 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:55:23.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:22 smithi110 ceph-mon[25392]: from='client.? 172.21.15.139:0/3248681975' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f9119e96-9863-45b7-b2d5-89bc3eeb1fed"}]: dispatch 2024-09-11T07:55:23.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:22 smithi110 ceph-mon[25392]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f9119e96-9863-45b7-b2d5-89bc3eeb1fed"}]: dispatch 2024-09-11T07:55:23.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:22 smithi110 ceph-mon[25392]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f9119e96-9863-45b7-b2d5-89bc3eeb1fed"}]': finished 2024-09-11T07:55:23.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:22 smithi110 ceph-mon[25392]: osdmap e27: 5 total, 4 up, 5 in 2024-09-11T07:55:23.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:22 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:55:23.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:22 smithi110 ceph-mon[31709]: from='client.? 172.21.15.139:0/3248681975' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f9119e96-9863-45b7-b2d5-89bc3eeb1fed"}]: dispatch 2024-09-11T07:55:23.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:22 smithi110 ceph-mon[31709]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f9119e96-9863-45b7-b2d5-89bc3eeb1fed"}]: dispatch 2024-09-11T07:55:23.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:22 smithi110 ceph-mon[31709]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f9119e96-9863-45b7-b2d5-89bc3eeb1fed"}]': finished 2024-09-11T07:55:23.219 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:22 smithi110 ceph-mon[31709]: osdmap e27: 5 total, 4 up, 5 in 2024-09-11T07:55:23.219 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:22 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:55:23.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:23 smithi139 ceph-mon[27311]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:55:23.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:23 smithi139 ceph-mon[27311]: from='client.? 172.21.15.139:0/2147021499' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:55:24.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:23 smithi110 ceph-mon[25392]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:55:24.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:23 smithi110 ceph-mon[25392]: from='client.? 172.21.15.139:0/2147021499' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:55:24.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:23 smithi110 ceph-mon[31709]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:55:24.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:23 smithi110 ceph-mon[31709]: from='client.? 172.21.15.139:0/2147021499' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:55:25.998 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:25 smithi139 ceph-mon[27311]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:55:26.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:25 smithi110 ceph-mon[25392]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:55:26.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:25 smithi110 ceph-mon[31709]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:55:27.941 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:27 smithi139 ceph-mon[27311]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:55:28.105 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:27 smithi110 ceph-mon[31709]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:55:28.105 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:27 smithi110 ceph-mon[25392]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:55:28.936 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:28 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-11T07:55:28.936 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:28 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:55:29.134 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:28 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-11T07:55:29.134 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:28 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:55:29.134 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:28 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-11T07:55:29.134 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:28 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:55:29.876 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:29 smithi139 ceph-mon[27311]: Deploying daemon osd.4 on smithi139 2024-09-11T07:55:29.876 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:29 smithi139 ceph-mon[27311]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:55:30.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:29 smithi110 ceph-mon[25392]: Deploying daemon osd.4 on smithi139 2024-09-11T07:55:30.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:29 smithi110 ceph-mon[25392]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:55:30.219 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:29 smithi110 ceph-mon[31709]: Deploying daemon osd.4 on smithi139 2024-09-11T07:55:30.219 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:29 smithi110 ceph-mon[31709]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:55:31.843 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:31 smithi139 ceph-mon[27311]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:55:31.843 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:31 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:31.843 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:31 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:31.843 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:31 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:31.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:31 smithi110 ceph-mon[25392]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:55:31.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:31 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:31.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:31 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:31.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:31 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:31.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:31 smithi110 ceph-mon[31709]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:55:31.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:31 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:31.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:31 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:31.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:31 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:33.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:33 smithi139 ceph-mon[27311]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:55:33.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:33 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:55:33.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:33 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:33.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:33 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:55:33.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:33 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:55:33.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:33 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:33.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:33 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:33.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:33 smithi110 ceph-mon[31709]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:55:33.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:33 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:55:33.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:33 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:33.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:33 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:55:33.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:33 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:55:33.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:33 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:33.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:33 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:33.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:33 smithi110 ceph-mon[25392]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:55:33.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:33 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:55:33.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:33 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:33.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:33 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:55:33.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:33 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:55:33.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:33 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:33.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:33 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:35.494 INFO:teuthology.orchestra.run.smithi139.stdout:Created osd(s) 4 on host 'smithi139' 2024-09-11T07:55:35.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:35 smithi139 ceph-mon[27311]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:55:35.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:35 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:55:35.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:35 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:35.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:35 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:55:35.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:35 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:35.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:35 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:35.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:35 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:55:35.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:35 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:55:35.750 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:35 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:35.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:35 smithi110 ceph-mon[25392]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:55:35.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:35 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:55:35.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:35 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:35.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:35 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:55:35.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:35 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:35.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:35 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:35.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:35 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:55:35.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:35 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:55:35.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:35 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:35.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:35 smithi110 ceph-mon[31709]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:55:35.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:35 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:55:35.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:35 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:35.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:35 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:55:35.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:35 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:35.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:35 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:35.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:35 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:55:35.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:35 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:55:35.970 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:35 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:36.011 DEBUG:teuthology.orchestra.run.smithi139:osd.4> sudo journalctl -f -n 0 -u ceph-41bd1096-7012-11ef-bcea-c7b262605968@osd.4.service 2024-09-11T07:55:36.014 INFO:tasks.cephadm:Deploying osd.5 on smithi139 with /dev/nvme2n1... 2024-09-11T07:55:36.014 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/nvme2n1 2024-09-11T07:55:36.288 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:55:36.388 INFO:journalctl@ceph.osd.4.smithi139.stdout:Sep 11 07:55:36 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-4[33592]: 2024-09-11T07:55:36.066+0000 7fd5ae945440 -1 osd.4 0 log_to_monitors true 2024-09-11T07:55:36.699 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:36 smithi139 ceph-mon[27311]: Adjusting osd_memory_target on smithi139 to 16064M 2024-09-11T07:55:36.699 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:36 smithi139 ceph-mon[27311]: from='osd.4 [v2:172.21.15.139:6800/2399332353,v1:172.21.15.139:6801/2399332353]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T07:55:36.699 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:36 smithi139 ceph-mon[27311]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T07:55:36.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:36 smithi110 ceph-mon[31709]: Adjusting osd_memory_target on smithi139 to 16064M 2024-09-11T07:55:36.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:36 smithi110 ceph-mon[31709]: from='osd.4 [v2:172.21.15.139:6800/2399332353,v1:172.21.15.139:6801/2399332353]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T07:55:36.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:36 smithi110 ceph-mon[31709]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T07:55:36.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:36 smithi110 ceph-mon[25392]: Adjusting osd_memory_target on smithi139 to 16064M 2024-09-11T07:55:36.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:36 smithi110 ceph-mon[25392]: from='osd.4 [v2:172.21.15.139:6800/2399332353,v1:172.21.15.139:6801/2399332353]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T07:55:36.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:36 smithi110 ceph-mon[25392]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T07:55:37.571 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:37 smithi139 ceph-mon[27311]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:55:37.571 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:37 smithi139 ceph-mon[27311]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-11T07:55:37.571 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:37 smithi139 ceph-mon[27311]: osdmap e28: 5 total, 4 up, 5 in 2024-09-11T07:55:37.571 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:37 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:55:37.571 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:37 smithi139 ceph-mon[27311]: from='osd.4 [v2:172.21.15.139:6800/2399332353,v1:172.21.15.139:6801/2399332353]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-09-11T07:55:37.571 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:37 smithi139 ceph-mon[27311]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-09-11T07:55:37.876 INFO:journalctl@ceph.osd.4.smithi139.stdout:Sep 11 07:55:37 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-4[33592]: 2024-09-11T07:55:37.569+0000 7fd5aa0b7640 -1 osd.4 0 waiting for initial osdmap 2024-09-11T07:55:37.877 INFO:journalctl@ceph.osd.4.smithi139.stdout:Sep 11 07:55:37 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-4[33592]: 2024-09-11T07:55:37.574+0000 7fd5a4eee640 -1 osd.4 29 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T07:55:37.899 INFO:teuthology.orchestra.run.smithi139.stderr:--> Zapping: /dev/nvme2n1 2024-09-11T07:55:37.899 INFO:teuthology.orchestra.run.smithi139.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-09-11T07:55:37.903 INFO:teuthology.orchestra.run.smithi139.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme2n1 bs=1M count=10 conv=fsync 2024-09-11T07:55:37.918 INFO:teuthology.orchestra.run.smithi139.stderr: stderr: 10+0 records in 2024-09-11T07:55:37.918 INFO:teuthology.orchestra.run.smithi139.stderr:10+0 records out 2024-09-11T07:55:37.918 INFO:teuthology.orchestra.run.smithi139.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0133031 s, 788 MB/s 2024-09-11T07:55:37.919 INFO:teuthology.orchestra.run.smithi139.stderr:--> Zapping successful for: 2024-09-11T07:55:37.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:37 smithi110 ceph-mon[25392]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:55:37.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:37 smithi110 ceph-mon[25392]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-11T07:55:37.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:37 smithi110 ceph-mon[25392]: osdmap e28: 5 total, 4 up, 5 in 2024-09-11T07:55:37.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:37 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:55:37.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:37 smithi110 ceph-mon[25392]: from='osd.4 [v2:172.21.15.139:6800/2399332353,v1:172.21.15.139:6801/2399332353]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-09-11T07:55:37.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:37 smithi110 ceph-mon[25392]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-09-11T07:55:37.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:37 smithi110 ceph-mon[31709]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:55:37.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:37 smithi110 ceph-mon[31709]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-11T07:55:37.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:37 smithi110 ceph-mon[31709]: osdmap e28: 5 total, 4 up, 5 in 2024-09-11T07:55:37.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:37 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:55:37.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:37 smithi110 ceph-mon[31709]: from='osd.4 [v2:172.21.15.139:6800/2399332353,v1:172.21.15.139:6801/2399332353]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-09-11T07:55:37.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:37 smithi110 ceph-mon[31709]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-09-11T07:55:38.467 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi139:/dev/nvme2n1 2024-09-11T07:55:38.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:38 smithi139 ceph-mon[27311]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-09-11T07:55:38.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:38 smithi139 ceph-mon[27311]: osdmap e29: 5 total, 4 up, 5 in 2024-09-11T07:55:38.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:38 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:55:38.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:38 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:55:38.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:38 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:38.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:38 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:38.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:38 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:55:38.750 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:38 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:38.750 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:38 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:55:38.750 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:38 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:55:38.750 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:38 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:38.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:38 smithi110 ceph-mon[25392]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-09-11T07:55:38.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:38 smithi110 ceph-mon[25392]: osdmap e29: 5 total, 4 up, 5 in 2024-09-11T07:55:38.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:38 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:55:38.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:38 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:55:38.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:38 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:38.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:38 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:38.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:38 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:55:38.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:38 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:38.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:38 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:55:38.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:38 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:55:38.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:38 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:38.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:38 smithi110 ceph-mon[31709]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-09-11T07:55:38.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:38 smithi110 ceph-mon[31709]: osdmap e29: 5 total, 4 up, 5 in 2024-09-11T07:55:38.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:38 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:55:38.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:38 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:55:38.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:38 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:38.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:38 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:38.970 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:38 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:55:38.970 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:38 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:38.970 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:38 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:55:38.970 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:38 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:55:38.970 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:38 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:39.077 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:55:39.855 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:39 smithi110 ceph-mon[25392]: purged_snaps scrub starts 2024-09-11T07:55:39.856 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:39 smithi110 ceph-mon[25392]: purged_snaps scrub ok 2024-09-11T07:55:39.856 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:39 smithi110 ceph-mon[25392]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:55:39.856 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:39 smithi110 ceph-mon[25392]: osd.4 [v2:172.21.15.139:6800/2399332353,v1:172.21.15.139:6801/2399332353] boot 2024-09-11T07:55:39.856 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:39 smithi110 ceph-mon[25392]: osdmap e30: 5 total, 5 up, 5 in 2024-09-11T07:55:39.856 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:39 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:55:39.856 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:39 smithi110 ceph-mon[25392]: osdmap e31: 5 total, 5 up, 5 in 2024-09-11T07:55:39.856 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:39 smithi110 ceph-mon[31709]: purged_snaps scrub starts 2024-09-11T07:55:39.856 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:39 smithi110 ceph-mon[31709]: purged_snaps scrub ok 2024-09-11T07:55:39.856 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:39 smithi110 ceph-mon[31709]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:55:39.856 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:39 smithi110 ceph-mon[31709]: osd.4 [v2:172.21.15.139:6800/2399332353,v1:172.21.15.139:6801/2399332353] boot 2024-09-11T07:55:39.856 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:39 smithi110 ceph-mon[31709]: osdmap e30: 5 total, 5 up, 5 in 2024-09-11T07:55:39.857 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:39 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:55:39.857 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:39 smithi110 ceph-mon[31709]: osdmap e31: 5 total, 5 up, 5 in 2024-09-11T07:55:39.876 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:39 smithi139 ceph-mon[27311]: purged_snaps scrub starts 2024-09-11T07:55:39.876 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:39 smithi139 ceph-mon[27311]: purged_snaps scrub ok 2024-09-11T07:55:39.876 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:39 smithi139 ceph-mon[27311]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:55:39.876 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:39 smithi139 ceph-mon[27311]: osd.4 [v2:172.21.15.139:6800/2399332353,v1:172.21.15.139:6801/2399332353] boot 2024-09-11T07:55:39.876 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:39 smithi139 ceph-mon[27311]: osdmap e30: 5 total, 5 up, 5 in 2024-09-11T07:55:39.876 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:39 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:55:39.877 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:39 smithi139 ceph-mon[27311]: osdmap e31: 5 total, 5 up, 5 in 2024-09-11T07:55:41.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:41 smithi139 ceph-mon[27311]: osdmap e32: 5 total, 5 up, 5 in 2024-09-11T07:55:41.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:41 smithi139 ceph-mon[27311]: pgmap v119: 1 pgs: 1 remapped+peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:55:41.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:41 smithi110 ceph-mon[25392]: osdmap e32: 5 total, 5 up, 5 in 2024-09-11T07:55:41.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:41 smithi110 ceph-mon[25392]: pgmap v119: 1 pgs: 1 remapped+peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:55:41.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:41 smithi110 ceph-mon[31709]: osdmap e32: 5 total, 5 up, 5 in 2024-09-11T07:55:41.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:41 smithi110 ceph-mon[31709]: pgmap v119: 1 pgs: 1 remapped+peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:55:42.434 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:42 smithi139 ceph-mon[27311]: osdmap e33: 5 total, 5 up, 5 in 2024-09-11T07:55:42.434 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:42 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:42.435 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:42 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:42.435 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:42 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:55:42.435 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:42 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:42.435 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:42 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:55:42.435 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:42 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:55:42.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:42 smithi110 ceph-mon[25392]: osdmap e33: 5 total, 5 up, 5 in 2024-09-11T07:55:42.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:42 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:42.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:42 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:42.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:42 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:55:42.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:42 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:42.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:42 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:55:42.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:42 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:55:42.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:42 smithi110 ceph-mon[31709]: osdmap e33: 5 total, 5 up, 5 in 2024-09-11T07:55:42.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:42 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:42.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:42 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:42.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:42 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:55:42.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:42 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:42.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:42 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:55:42.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:42 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:55:43.498 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:43 smithi139 ceph-mon[27311]: Detected new or changed devices on smithi139 2024-09-11T07:55:43.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:43 smithi139 ceph-mon[27311]: osdmap e34: 5 total, 5 up, 5 in 2024-09-11T07:55:43.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:43 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:43.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:43 smithi139 ceph-mon[27311]: pgmap v122: 1 pgs: 1 remapped+peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:55:43.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:43 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:55:43.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:43 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:55:43.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:43 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:55:43.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:43 smithi110 ceph-mon[31709]: Detected new or changed devices on smithi139 2024-09-11T07:55:43.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:43 smithi110 ceph-mon[31709]: osdmap e34: 5 total, 5 up, 5 in 2024-09-11T07:55:43.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:43 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:43.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:43 smithi110 ceph-mon[31709]: pgmap v122: 1 pgs: 1 remapped+peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:55:43.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:43 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:55:43.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:43 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:55:43.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:43 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:55:43.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:43 smithi110 ceph-mon[25392]: Detected new or changed devices on smithi139 2024-09-11T07:55:43.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:43 smithi110 ceph-mon[25392]: osdmap e34: 5 total, 5 up, 5 in 2024-09-11T07:55:43.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:43 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:43.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:43 smithi110 ceph-mon[25392]: pgmap v122: 1 pgs: 1 remapped+peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:55:43.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:43 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:55:43.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:43 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:55:43.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:43 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:55:44.498 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:44 smithi139 ceph-mon[27311]: from='client.24205 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:/dev/nvme2n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:55:44.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:44 smithi110 ceph-mon[25392]: from='client.24205 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:/dev/nvme2n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:55:44.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:44 smithi110 ceph-mon[31709]: from='client.24205 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:/dev/nvme2n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:55:45.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:45 smithi110 ceph-mon[25392]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-09-11T07:55:45.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:45 smithi110 ceph-mon[25392]: from='client.? 172.21.15.139:0/2026926638' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3339eb92-7017-4bb8-947f-557a32723e94"}]: dispatch 2024-09-11T07:55:45.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:45 smithi110 ceph-mon[25392]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3339eb92-7017-4bb8-947f-557a32723e94"}]: dispatch 2024-09-11T07:55:45.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:45 smithi110 ceph-mon[25392]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3339eb92-7017-4bb8-947f-557a32723e94"}]': finished 2024-09-11T07:55:45.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:45 smithi110 ceph-mon[25392]: osdmap e35: 6 total, 5 up, 6 in 2024-09-11T07:55:45.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:45 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:55:45.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:45 smithi110 ceph-mon[31709]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-09-11T07:55:45.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:45 smithi110 ceph-mon[31709]: from='client.? 172.21.15.139:0/2026926638' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3339eb92-7017-4bb8-947f-557a32723e94"}]: dispatch 2024-09-11T07:55:45.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:45 smithi110 ceph-mon[31709]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3339eb92-7017-4bb8-947f-557a32723e94"}]: dispatch 2024-09-11T07:55:45.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:45 smithi110 ceph-mon[31709]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3339eb92-7017-4bb8-947f-557a32723e94"}]': finished 2024-09-11T07:55:45.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:45 smithi110 ceph-mon[31709]: osdmap e35: 6 total, 5 up, 6 in 2024-09-11T07:55:45.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:45 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:55:45.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:45 smithi139 ceph-mon[27311]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-09-11T07:55:45.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:45 smithi139 ceph-mon[27311]: from='client.? 172.21.15.139:0/2026926638' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3339eb92-7017-4bb8-947f-557a32723e94"}]: dispatch 2024-09-11T07:55:45.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:45 smithi139 ceph-mon[27311]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3339eb92-7017-4bb8-947f-557a32723e94"}]: dispatch 2024-09-11T07:55:45.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:45 smithi139 ceph-mon[27311]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3339eb92-7017-4bb8-947f-557a32723e94"}]': finished 2024-09-11T07:55:45.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:45 smithi139 ceph-mon[27311]: osdmap e35: 6 total, 5 up, 6 in 2024-09-11T07:55:45.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:45 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:55:46.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:46 smithi110 ceph-mon[25392]: from='client.? 172.21.15.139:0/942147334' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:55:46.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:46 smithi110 ceph-mon[31709]: from='client.? 172.21.15.139:0/942147334' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:55:46.748 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:46 smithi139 ceph-mon[27311]: from='client.? 172.21.15.139:0/942147334' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:55:47.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:47 smithi110 ceph-mon[31709]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T07:55:47.556 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:47 smithi110 ceph-mon[25392]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T07:55:47.748 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:47 smithi139 ceph-mon[27311]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T07:55:49.498 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:49 smithi139 ceph-mon[27311]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-11T07:55:49.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:49 smithi110 ceph-mon[25392]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-11T07:55:49.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:49 smithi110 ceph-mon[31709]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-11T07:55:51.865 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:51 smithi139 ceph-mon[27311]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-09-11T07:55:51.865 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:51 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:51.866 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:51 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:51.866 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:51 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:51.866 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:51 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-11T07:55:51.866 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:51 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:55:51.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:51 smithi110 ceph-mon[25392]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-09-11T07:55:51.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:51 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:51.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:51 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:51.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:51 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:51.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:51 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-11T07:55:51.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:51 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:55:51.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:51 smithi110 ceph-mon[31709]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-09-11T07:55:51.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:51 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:51.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:51 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:51.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:51 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:51.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:51 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-11T07:55:51.970 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:51 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:55:52.946 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:52 smithi139 ceph-mon[27311]: Deploying daemon osd.5 on smithi139 2024-09-11T07:55:52.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:52 smithi110 ceph-mon[31709]: Deploying daemon osd.5 on smithi139 2024-09-11T07:55:52.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:52 smithi110 ceph-mon[25392]: Deploying daemon osd.5 on smithi139 2024-09-11T07:55:53.874 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:53 smithi139 ceph-mon[27311]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T07:55:53.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:53 smithi110 ceph-mon[25392]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T07:55:53.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:53 smithi110 ceph-mon[31709]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T07:55:55.959 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:55 smithi139 ceph-mon[27311]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:55:55.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:55 smithi110 ceph-mon[25392]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:55:55.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:55 smithi110 ceph-mon[31709]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:55:56.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:56 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:55:56.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:56 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:56.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:56 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:55:56.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:56 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:55:56.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:56 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:56.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:56 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:56.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:56 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:55:56.970 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:56 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:56.970 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:56 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:55:56.970 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:56 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:55:56.970 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:56 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:56.970 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:56 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:56.998 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:56 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:55:56.998 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:56 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:56.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:56 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:55:56.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:56 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:55:56.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:56 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:56.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:56 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:57.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:57 smithi110 ceph-mon[25392]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:55:57.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:57 smithi110 ceph-mon[31709]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:55:57.998 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:57 smithi139 ceph-mon[27311]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:55:58.623 INFO:teuthology.orchestra.run.smithi139.stdout:Created osd(s) 5 on host 'smithi139' 2024-09-11T07:55:58.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:58 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:55:58.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:58 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:58.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:58 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:55:58.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:58 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:55:58.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:58 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:58.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:58 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:58.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:58 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:55:58.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:58 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:55:58.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:58 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:58.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:58 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:55:58.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:58 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:58.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:58 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:55:58.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:58 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:55:58.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:58 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:58.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:58 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:58.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:58 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:55:58.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:58 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:55:58.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:58 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:58.998 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:58 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:55:58.998 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:58 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:58.998 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:58 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:55:58.998 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:58 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:55:58.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:58 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:58.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:58 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:58.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:58 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:55:58.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:58 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:55:58.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:58 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:55:59.228 DEBUG:teuthology.orchestra.run.smithi139:osd.5> sudo journalctl -f -n 0 -u ceph-41bd1096-7012-11ef-bcea-c7b262605968@osd.5.service 2024-09-11T07:55:59.230 INFO:tasks.cephadm:Deploying osd.6 on smithi139 with /dev/nvme3n1... 2024-09-11T07:55:59.230 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/nvme3n1 2024-09-11T07:55:59.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:59 smithi110 ceph-mon[31709]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:55:59.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:59 smithi110 ceph-mon[31709]: Adjusting osd_memory_target on smithi139 to 8032M 2024-09-11T07:55:59.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:59 smithi110 ceph-mon[31709]: from='osd.5 [v2:172.21.15.139:6808/1641418305,v1:172.21.15.139:6809/1641418305]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T07:55:59.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:55:59 smithi110 ceph-mon[31709]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T07:55:59.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:59 smithi110 ceph-mon[25392]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:55:59.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:59 smithi110 ceph-mon[25392]: Adjusting osd_memory_target on smithi139 to 8032M 2024-09-11T07:55:59.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:59 smithi110 ceph-mon[25392]: from='osd.5 [v2:172.21.15.139:6808/1641418305,v1:172.21.15.139:6809/1641418305]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T07:55:59.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:55:59 smithi110 ceph-mon[25392]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T07:55:59.998 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:59 smithi139 ceph-mon[27311]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:55:59.998 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:59 smithi139 ceph-mon[27311]: Adjusting osd_memory_target on smithi139 to 8032M 2024-09-11T07:55:59.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:59 smithi139 ceph-mon[27311]: from='osd.5 [v2:172.21.15.139:6808/1641418305,v1:172.21.15.139:6809/1641418305]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T07:55:59.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:55:59 smithi139 ceph-mon[27311]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T07:56:00.172 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:56:00.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:00 smithi110 ceph-mon[31709]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-11T07:56:00.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:00 smithi110 ceph-mon[31709]: osdmap e36: 6 total, 5 up, 6 in 2024-09-11T07:56:00.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:00 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:56:00.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:00 smithi110 ceph-mon[31709]: from='osd.5 [v2:172.21.15.139:6808/1641418305,v1:172.21.15.139:6809/1641418305]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-09-11T07:56:00.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:00 smithi110 ceph-mon[31709]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-09-11T07:56:00.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:00 smithi110 ceph-mon[25392]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-11T07:56:00.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:00 smithi110 ceph-mon[25392]: osdmap e36: 6 total, 5 up, 6 in 2024-09-11T07:56:00.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:00 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:56:00.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:00 smithi110 ceph-mon[25392]: from='osd.5 [v2:172.21.15.139:6808/1641418305,v1:172.21.15.139:6809/1641418305]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-09-11T07:56:00.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:00 smithi110 ceph-mon[25392]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-09-11T07:56:00.998 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:00 smithi139 ceph-mon[27311]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-11T07:56:00.998 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:00 smithi139 ceph-mon[27311]: osdmap e36: 6 total, 5 up, 6 in 2024-09-11T07:56:00.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:00 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:56:00.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:00 smithi139 ceph-mon[27311]: from='osd.5 [v2:172.21.15.139:6808/1641418305,v1:172.21.15.139:6809/1641418305]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-09-11T07:56:00.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:00 smithi139 ceph-mon[27311]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-09-11T07:56:00.999 INFO:journalctl@ceph.osd.5.smithi139.stdout:Sep 11 07:56:00 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-5[37043]: 2024-09-11T07:56:00.708+0000 7f257f4d0640 -1 osd.5 0 waiting for initial osdmap 2024-09-11T07:56:00.999 INFO:journalctl@ceph.osd.5.smithi139.stdout:Sep 11 07:56:00 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-5[37043]: 2024-09-11T07:56:00.713+0000 7f2578ae2640 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T07:56:01.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:01 smithi110 ceph-mon[25392]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:56:01.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:01 smithi110 ceph-mon[25392]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-09-11T07:56:01.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:01 smithi110 ceph-mon[25392]: osdmap e37: 6 total, 5 up, 6 in 2024-09-11T07:56:01.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:01 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:56:01.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:01 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:56:01.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:01 smithi110 ceph-mon[31709]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:56:01.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:01 smithi110 ceph-mon[31709]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-09-11T07:56:01.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:01 smithi110 ceph-mon[31709]: osdmap e37: 6 total, 5 up, 6 in 2024-09-11T07:56:01.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:01 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:56:01.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:01 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:56:01.998 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:01 smithi139 ceph-mon[27311]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:56:01.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:01 smithi139 ceph-mon[27311]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-09-11T07:56:01.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:01 smithi139 ceph-mon[27311]: osdmap e37: 6 total, 5 up, 6 in 2024-09-11T07:56:01.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:01 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:56:01.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:01 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:56:02.925 INFO:teuthology.orchestra.run.smithi139.stderr:--> Zapping: /dev/nvme3n1 2024-09-11T07:56:02.925 INFO:teuthology.orchestra.run.smithi139.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-09-11T07:56:02.930 INFO:teuthology.orchestra.run.smithi139.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme3n1 bs=1M count=10 conv=fsync 2024-09-11T07:56:02.945 INFO:teuthology.orchestra.run.smithi139.stderr: stderr: 10+0 records in 2024-09-11T07:56:02.945 INFO:teuthology.orchestra.run.smithi139.stderr:10+0 records out 2024-09-11T07:56:02.945 INFO:teuthology.orchestra.run.smithi139.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0133215 s, 787 MB/s 2024-09-11T07:56:02.945 INFO:teuthology.orchestra.run.smithi139.stderr:--> Zapping successful for: 2024-09-11T07:56:02.965 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:02 smithi139 ceph-mon[27311]: purged_snaps scrub starts 2024-09-11T07:56:02.965 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:02 smithi139 ceph-mon[27311]: purged_snaps scrub ok 2024-09-11T07:56:02.965 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:02 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:56:02.965 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:02 smithi139 ceph-mon[27311]: osd.5 [v2:172.21.15.139:6808/1641418305,v1:172.21.15.139:6809/1641418305] boot 2024-09-11T07:56:02.965 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:02 smithi139 ceph-mon[27311]: osdmap e38: 6 total, 6 up, 6 in 2024-09-11T07:56:02.965 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:02 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:56:02.965 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:02 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:02.965 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:02 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:02.966 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:02 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:56:02.966 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:02 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:02.966 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:02 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:56:02.966 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:02 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:56:02.966 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:02 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:02.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:02 smithi110 ceph-mon[25392]: purged_snaps scrub starts 2024-09-11T07:56:02.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:02 smithi110 ceph-mon[25392]: purged_snaps scrub ok 2024-09-11T07:56:02.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:02 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:56:02.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:02 smithi110 ceph-mon[25392]: osd.5 [v2:172.21.15.139:6808/1641418305,v1:172.21.15.139:6809/1641418305] boot 2024-09-11T07:56:02.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:02 smithi110 ceph-mon[25392]: osdmap e38: 6 total, 6 up, 6 in 2024-09-11T07:56:02.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:02 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:56:02.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:02 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:02.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:02 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:02.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:02 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:56:02.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:02 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:02.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:02 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:56:02.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:02 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:56:02.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:02 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:02.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:02 smithi110 ceph-mon[31709]: purged_snaps scrub starts 2024-09-11T07:56:02.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:02 smithi110 ceph-mon[31709]: purged_snaps scrub ok 2024-09-11T07:56:02.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:02 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:56:02.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:02 smithi110 ceph-mon[31709]: osd.5 [v2:172.21.15.139:6808/1641418305,v1:172.21.15.139:6809/1641418305] boot 2024-09-11T07:56:02.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:02 smithi110 ceph-mon[31709]: osdmap e38: 6 total, 6 up, 6 in 2024-09-11T07:56:02.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:02 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:56:02.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:02 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:02.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:02 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:02.970 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:02 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:56:02.970 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:02 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:02.970 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:02 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:56:02.970 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:02 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:56:02.970 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:02 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:03.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:03 smithi110 ceph-mon[25392]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:56:03.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:03 smithi110 ceph-mon[25392]: Detected new or changed devices on smithi139 2024-09-11T07:56:03.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:03 smithi110 ceph-mon[25392]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:56:03.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:03 smithi110 ceph-mon[25392]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-09-11T07:56:03.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:03 smithi110 ceph-mon[25392]: osdmap e39: 6 total, 6 up, 6 in 2024-09-11T07:56:03.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:03 smithi110 ceph-mon[31709]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:56:03.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:03 smithi110 ceph-mon[31709]: Detected new or changed devices on smithi139 2024-09-11T07:56:03.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:03 smithi110 ceph-mon[31709]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:56:03.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:03 smithi110 ceph-mon[31709]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-09-11T07:56:03.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:03 smithi110 ceph-mon[31709]: osdmap e39: 6 total, 6 up, 6 in 2024-09-11T07:56:03.998 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:03 smithi139 ceph-mon[27311]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:56:03.998 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:03 smithi139 ceph-mon[27311]: Detected new or changed devices on smithi139 2024-09-11T07:56:03.998 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:03 smithi139 ceph-mon[27311]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:56:03.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:03 smithi139 ceph-mon[27311]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-09-11T07:56:03.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:03 smithi139 ceph-mon[27311]: osdmap e39: 6 total, 6 up, 6 in 2024-09-11T07:56:04.113 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi139:/dev/nvme3n1 2024-09-11T07:56:04.362 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:56:04.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:04 smithi139 ceph-mon[27311]: osdmap e40: 6 total, 6 up, 6 in 2024-09-11T07:56:04.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:04 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:04.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:04 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:04.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:04 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:56:04.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:04 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:04.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:04 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:56:04.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:04 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:56:04.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:04 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:04.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:04 smithi139 ceph-mon[27311]: osdmap e41: 6 total, 6 up, 6 in 2024-09-11T07:56:04.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:04 smithi110 ceph-mon[31709]: osdmap e40: 6 total, 6 up, 6 in 2024-09-11T07:56:04.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:04 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:04.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:04 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:04.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:04 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:56:04.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:04 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:04.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:04 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:56:04.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:04 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:56:04.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:04 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:04.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:04 smithi110 ceph-mon[31709]: osdmap e41: 6 total, 6 up, 6 in 2024-09-11T07:56:04.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:04 smithi110 ceph-mon[25392]: osdmap e40: 6 total, 6 up, 6 in 2024-09-11T07:56:04.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:04 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:04.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:04 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:04.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:04 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:56:04.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:04 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:04.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:04 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:56:04.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:04 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:56:04.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:04 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:04.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:04 smithi110 ceph-mon[25392]: osdmap e41: 6 total, 6 up, 6 in 2024-09-11T07:56:05.731 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:05 smithi139 ceph-mon[27311]: pgmap v140: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:56:05.731 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:05 smithi139 ceph-mon[27311]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2024-09-11T07:56:05.731 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:05 smithi139 ceph-mon[27311]: Cluster is now healthy 2024-09-11T07:56:05.731 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:05 smithi139 ceph-mon[27311]: osdmap e42: 6 total, 6 up, 6 in 2024-09-11T07:56:06.002 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:05 smithi110 ceph-mon[31709]: pgmap v140: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:56:06.002 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:05 smithi110 ceph-mon[31709]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2024-09-11T07:56:06.002 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:05 smithi110 ceph-mon[31709]: Cluster is now healthy 2024-09-11T07:56:06.002 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:05 smithi110 ceph-mon[31709]: osdmap e42: 6 total, 6 up, 6 in 2024-09-11T07:56:06.002 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:05 smithi110 ceph-mon[25392]: pgmap v140: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:56:06.002 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:05 smithi110 ceph-mon[25392]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2024-09-11T07:56:06.003 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:05 smithi110 ceph-mon[25392]: Cluster is now healthy 2024-09-11T07:56:06.003 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:05 smithi110 ceph-mon[25392]: osdmap e42: 6 total, 6 up, 6 in 2024-09-11T07:56:06.944 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:06 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:56:06.944 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:06 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:56:06.945 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:06 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:56:07.081 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:06 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:56:07.081 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:06 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:56:07.081 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:06 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:56:07.082 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:06 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:56:07.082 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:06 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:56:07.082 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:06 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:56:07.748 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:07 smithi139 ceph-mon[27311]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:/dev/nvme3n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:56:07.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:07 smithi139 ceph-mon[27311]: pgmap v143: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:56:08.120 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:07 smithi110 ceph-mon[25392]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:/dev/nvme3n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:56:08.121 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:07 smithi110 ceph-mon[25392]: pgmap v143: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:56:08.121 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:07 smithi110 ceph-mon[31709]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:/dev/nvme3n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:56:08.121 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:07 smithi110 ceph-mon[31709]: pgmap v143: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:56:08.998 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:08 smithi139 ceph-mon[27311]: from='client.? 172.21.15.139:0/2209949851' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "69a4457c-39cc-41f5-bceb-9229b8b030f7"}]: dispatch 2024-09-11T07:56:08.998 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:08 smithi139 ceph-mon[27311]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "69a4457c-39cc-41f5-bceb-9229b8b030f7"}]: dispatch 2024-09-11T07:56:08.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:08 smithi139 ceph-mon[27311]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "69a4457c-39cc-41f5-bceb-9229b8b030f7"}]': finished 2024-09-11T07:56:08.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:08 smithi139 ceph-mon[27311]: osdmap e43: 7 total, 6 up, 7 in 2024-09-11T07:56:08.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:08 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:56:09.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:08 smithi110 ceph-mon[25392]: from='client.? 172.21.15.139:0/2209949851' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "69a4457c-39cc-41f5-bceb-9229b8b030f7"}]: dispatch 2024-09-11T07:56:09.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:08 smithi110 ceph-mon[25392]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "69a4457c-39cc-41f5-bceb-9229b8b030f7"}]: dispatch 2024-09-11T07:56:09.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:08 smithi110 ceph-mon[25392]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "69a4457c-39cc-41f5-bceb-9229b8b030f7"}]': finished 2024-09-11T07:56:09.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:08 smithi110 ceph-mon[25392]: osdmap e43: 7 total, 6 up, 7 in 2024-09-11T07:56:09.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:08 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:56:09.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:08 smithi110 ceph-mon[31709]: from='client.? 172.21.15.139:0/2209949851' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "69a4457c-39cc-41f5-bceb-9229b8b030f7"}]: dispatch 2024-09-11T07:56:09.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:08 smithi110 ceph-mon[31709]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "69a4457c-39cc-41f5-bceb-9229b8b030f7"}]: dispatch 2024-09-11T07:56:09.219 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:08 smithi110 ceph-mon[31709]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "69a4457c-39cc-41f5-bceb-9229b8b030f7"}]': finished 2024-09-11T07:56:09.219 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:08 smithi110 ceph-mon[31709]: osdmap e43: 7 total, 6 up, 7 in 2024-09-11T07:56:09.219 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:08 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:56:09.998 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:09 smithi139 ceph-mon[27311]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:56:09.998 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:09 smithi139 ceph-mon[27311]: from='client.? 172.21.15.139:0/3270852292' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:56:10.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:09 smithi110 ceph-mon[31709]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:56:10.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:09 smithi110 ceph-mon[31709]: from='client.? 172.21.15.139:0/3270852292' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:56:10.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:09 smithi110 ceph-mon[25392]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:56:10.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:09 smithi110 ceph-mon[25392]: from='client.? 172.21.15.139:0/3270852292' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:56:11.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:11 smithi139 ceph-mon[27311]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:56:11.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:11 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:11.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:11 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:11.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:11 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:12.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:11 smithi110 ceph-mon[25392]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:56:12.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:11 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:12.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:11 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:12.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:11 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:12.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:11 smithi110 ceph-mon[31709]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:56:12.219 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:11 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:12.219 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:11 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:12.219 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:11 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:13.998 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:13 smithi139 ceph-mon[27311]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:56:14.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:13 smithi110 ceph-mon[31709]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:56:14.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:13 smithi110 ceph-mon[25392]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:56:15.866 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:15 smithi139 ceph-mon[27311]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:56:15.866 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:15 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-11T07:56:15.866 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:15 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:56:16.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:15 smithi110 ceph-mon[31709]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:56:16.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:15 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-11T07:56:16.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:15 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:56:16.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:15 smithi110 ceph-mon[25392]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:56:16.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:15 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-11T07:56:16.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:15 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:56:16.997 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:16 smithi139 ceph-mon[27311]: Deploying daemon osd.6 on smithi139 2024-09-11T07:56:17.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:16 smithi110 ceph-mon[31709]: Deploying daemon osd.6 on smithi139 2024-09-11T07:56:17.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:16 smithi110 ceph-mon[25392]: Deploying daemon osd.6 on smithi139 2024-09-11T07:56:17.933 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:17 smithi139 ceph-mon[27311]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:56:18.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:17 smithi110 ceph-mon[31709]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:56:18.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:17 smithi110 ceph-mon[25392]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:56:19.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:19 smithi139 ceph-mon[27311]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:56:20.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:19 smithi110 ceph-mon[25392]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:56:20.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:19 smithi110 ceph-mon[31709]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:56:21.038 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:20 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:56:21.038 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:20 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:21.039 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:20 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:56:21.039 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:20 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:56:21.039 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:20 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:21.039 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:20 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:21.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:20 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:56:21.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:20 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:21.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:20 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:56:21.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:20 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:56:21.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:20 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:21.219 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:20 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:21.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:20 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:56:21.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:20 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:21.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:20 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:56:21.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:20 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:56:21.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:20 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:21.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:20 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:22.025 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:21 smithi139 ceph-mon[27311]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:56:22.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:21 smithi110 ceph-mon[25392]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:56:22.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:21 smithi110 ceph-mon[31709]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:56:23.809 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:23 smithi139 ceph-mon[27311]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:56:23.809 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:23 smithi139 ceph-mon[27311]: from='osd.6 [v2:172.21.15.139:6816/1783109268,v1:172.21.15.139:6817/1783109268]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T07:56:23.809 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:23 smithi139 ceph-mon[27311]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T07:56:24.119 INFO:teuthology.orchestra.run.smithi139.stdout:Created osd(s) 6 on host 'smithi139' 2024-09-11T07:56:24.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:23 smithi110 ceph-mon[31709]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:56:24.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:23 smithi110 ceph-mon[31709]: from='osd.6 [v2:172.21.15.139:6816/1783109268,v1:172.21.15.139:6817/1783109268]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T07:56:24.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:23 smithi110 ceph-mon[31709]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T07:56:24.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:23 smithi110 ceph-mon[25392]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:56:24.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:23 smithi110 ceph-mon[25392]: from='osd.6 [v2:172.21.15.139:6816/1783109268,v1:172.21.15.139:6817/1783109268]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T07:56:24.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:23 smithi110 ceph-mon[25392]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T07:56:24.678 DEBUG:teuthology.orchestra.run.smithi139:osd.6> sudo journalctl -f -n 0 -u ceph-41bd1096-7012-11ef-bcea-c7b262605968@osd.6.service 2024-09-11T07:56:24.680 INFO:tasks.cephadm:Deploying osd.7 on smithi139 with /dev/nvme4n1... 2024-09-11T07:56:24.680 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/nvme4n1 2024-09-11T07:56:24.950 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 07:56:24 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6[40580]: 2024-09-11T07:56:24.767+0000 7f7cb7829640 -1 osd.6 0 waiting for initial osdmap 2024-09-11T07:56:24.950 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 07:56:24 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6[40580]: 2024-09-11T07:56:24.774+0000 7f7cb065c640 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T07:56:24.950 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:24 smithi139 ceph-mon[27311]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-11T07:56:24.950 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:24 smithi139 ceph-mon[27311]: osdmap e44: 7 total, 6 up, 7 in 2024-09-11T07:56:24.951 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:24 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:56:24.951 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:24 smithi139 ceph-mon[27311]: from='osd.6 [v2:172.21.15.139:6816/1783109268,v1:172.21.15.139:6817/1783109268]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-09-11T07:56:24.951 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:24 smithi139 ceph-mon[27311]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-09-11T07:56:24.951 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:24 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:56:24.951 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:24 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:24.951 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:24 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:56:24.951 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:24 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:56:24.951 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:24 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:24.951 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:24 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:56:24.951 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:24 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:24.951 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:24 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:56:24.951 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:24 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:56:24.951 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:24 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:24.952 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:24 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:24.952 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:24 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:24.952 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:24 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:56:24.952 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:24 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:24.952 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:24 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:56:24.952 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:24 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:56:24.952 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:24 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:25.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:24 smithi110 ceph-mon[25392]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-11T07:56:25.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:24 smithi110 ceph-mon[25392]: osdmap e44: 7 total, 6 up, 7 in 2024-09-11T07:56:25.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:24 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:56:25.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:24 smithi110 ceph-mon[25392]: from='osd.6 [v2:172.21.15.139:6816/1783109268,v1:172.21.15.139:6817/1783109268]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-09-11T07:56:25.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:24 smithi110 ceph-mon[25392]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-09-11T07:56:25.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:24 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:56:25.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:24 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:25.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:24 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:56:25.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:24 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:56:25.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:24 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:25.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:24 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:56:25.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:24 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:25.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:24 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:56:25.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:24 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:56:25.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:24 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:25.220 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:24 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:25.220 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:24 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:25.220 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:24 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:56:25.220 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:24 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:25.220 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:24 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:56:25.220 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:24 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:56:25.220 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:24 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:25.220 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:24 smithi110 ceph-mon[31709]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-11T07:56:25.220 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:24 smithi110 ceph-mon[31709]: osdmap e44: 7 total, 6 up, 7 in 2024-09-11T07:56:25.221 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:24 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:56:25.221 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:24 smithi110 ceph-mon[31709]: from='osd.6 [v2:172.21.15.139:6816/1783109268,v1:172.21.15.139:6817/1783109268]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-09-11T07:56:25.221 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:24 smithi110 ceph-mon[31709]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-09-11T07:56:25.221 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:24 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:56:25.221 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:24 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:25.221 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:24 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:56:25.221 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:24 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:56:25.221 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:24 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:25.221 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:24 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:56:25.221 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:24 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:25.221 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:24 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:56:25.221 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:24 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:56:25.222 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:24 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:25.222 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:24 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:25.222 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:24 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:25.222 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:24 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:56:25.222 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:24 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:25.222 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:24 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:56:25.222 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:24 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:56:25.222 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:24 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:25.362 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:56:26.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:25 smithi110 ceph-mon[25392]: Adjusting osd_memory_target on smithi139 to 5354M 2024-09-11T07:56:26.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:25 smithi110 ceph-mon[25392]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:56:26.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:25 smithi110 ceph-mon[25392]: Detected new or changed devices on smithi139 2024-09-11T07:56:26.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:25 smithi110 ceph-mon[25392]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-09-11T07:56:26.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:25 smithi110 ceph-mon[25392]: osdmap e45: 7 total, 6 up, 7 in 2024-09-11T07:56:26.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:25 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:56:26.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:25 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:56:26.219 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:25 smithi110 ceph-mon[31709]: Adjusting osd_memory_target on smithi139 to 5354M 2024-09-11T07:56:26.219 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:25 smithi110 ceph-mon[31709]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:56:26.219 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:25 smithi110 ceph-mon[31709]: Detected new or changed devices on smithi139 2024-09-11T07:56:26.219 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:25 smithi110 ceph-mon[31709]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-09-11T07:56:26.219 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:25 smithi110 ceph-mon[31709]: osdmap e45: 7 total, 6 up, 7 in 2024-09-11T07:56:26.220 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:25 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:56:26.220 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:25 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:56:26.248 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:25 smithi139 ceph-mon[27311]: Adjusting osd_memory_target on smithi139 to 5354M 2024-09-11T07:56:26.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:25 smithi139 ceph-mon[27311]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:56:26.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:25 smithi139 ceph-mon[27311]: Detected new or changed devices on smithi139 2024-09-11T07:56:26.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:25 smithi139 ceph-mon[27311]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-09-11T07:56:26.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:25 smithi139 ceph-mon[27311]: osdmap e45: 7 total, 6 up, 7 in 2024-09-11T07:56:26.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:25 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:56:26.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:25 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:56:26.823 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:26 smithi110 ceph-mon[25392]: purged_snaps scrub starts 2024-09-11T07:56:26.824 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:26 smithi110 ceph-mon[25392]: purged_snaps scrub ok 2024-09-11T07:56:26.824 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:26 smithi110 ceph-mon[25392]: osd.6 [v2:172.21.15.139:6816/1783109268,v1:172.21.15.139:6817/1783109268] boot 2024-09-11T07:56:26.824 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:26 smithi110 ceph-mon[25392]: osdmap e46: 7 total, 7 up, 7 in 2024-09-11T07:56:26.824 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:26 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:56:26.824 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:26 smithi110 ceph-mon[31709]: purged_snaps scrub starts 2024-09-11T07:56:26.824 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:26 smithi110 ceph-mon[31709]: purged_snaps scrub ok 2024-09-11T07:56:26.824 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:26 smithi110 ceph-mon[31709]: osd.6 [v2:172.21.15.139:6816/1783109268,v1:172.21.15.139:6817/1783109268] boot 2024-09-11T07:56:26.824 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:26 smithi110 ceph-mon[31709]: osdmap e46: 7 total, 7 up, 7 in 2024-09-11T07:56:26.824 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:26 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:56:26.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:26 smithi139 ceph-mon[27311]: purged_snaps scrub starts 2024-09-11T07:56:26.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:26 smithi139 ceph-mon[27311]: purged_snaps scrub ok 2024-09-11T07:56:26.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:26 smithi139 ceph-mon[27311]: osd.6 [v2:172.21.15.139:6816/1783109268,v1:172.21.15.139:6817/1783109268] boot 2024-09-11T07:56:26.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:26 smithi139 ceph-mon[27311]: osdmap e46: 7 total, 7 up, 7 in 2024-09-11T07:56:26.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:26 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:56:27.062 INFO:teuthology.orchestra.run.smithi139.stderr:--> Zapping: /dev/nvme4n1 2024-09-11T07:56:27.063 INFO:teuthology.orchestra.run.smithi139.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-09-11T07:56:27.067 INFO:teuthology.orchestra.run.smithi139.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme4n1 bs=1M count=10 conv=fsync 2024-09-11T07:56:27.083 INFO:teuthology.orchestra.run.smithi139.stderr: stderr: 10+0 records in 2024-09-11T07:56:27.084 INFO:teuthology.orchestra.run.smithi139.stderr:10+0 records out 2024-09-11T07:56:27.084 INFO:teuthology.orchestra.run.smithi139.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0143207 s, 732 MB/s 2024-09-11T07:56:27.084 INFO:teuthology.orchestra.run.smithi139.stderr:--> Zapping successful for: 2024-09-11T07:56:27.697 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi139:/dev/nvme4n1 2024-09-11T07:56:27.940 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:56:27.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:27 smithi110 ceph-mon[25392]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:56:27.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:27 smithi110 ceph-mon[25392]: osdmap e47: 7 total, 7 up, 7 in 2024-09-11T07:56:27.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:27 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:27.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:27 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:27.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:27 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:56:27.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:27 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:27.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:27 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:56:27.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:27 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:56:27.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:27 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:27.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:27 smithi110 ceph-mon[31709]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:56:27.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:27 smithi110 ceph-mon[31709]: osdmap e47: 7 total, 7 up, 7 in 2024-09-11T07:56:27.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:27 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:27.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:27 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:27.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:27 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:56:27.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:27 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:27.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:27 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:56:27.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:27 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:56:27.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:27 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:28.062 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:27 smithi139 ceph-mon[27311]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:56:28.062 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:27 smithi139 ceph-mon[27311]: osdmap e47: 7 total, 7 up, 7 in 2024-09-11T07:56:28.062 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:27 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:28.062 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:27 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:28.062 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:27 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:56:28.062 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:27 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:28.062 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:27 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:56:28.062 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:27 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:56:28.062 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:27 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:28.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:28 smithi110 ceph-mon[25392]: osdmap e48: 7 total, 7 up, 7 in 2024-09-11T07:56:28.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:28 smithi110 ceph-mon[31709]: osdmap e48: 7 total, 7 up, 7 in 2024-09-11T07:56:28.998 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:28 smithi139 ceph-mon[27311]: osdmap e48: 7 total, 7 up, 7 in 2024-09-11T07:56:30.074 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:29 smithi139 ceph-mon[27311]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:56:30.074 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:29 smithi139 ceph-mon[27311]: osdmap e49: 7 total, 7 up, 7 in 2024-09-11T07:56:30.074 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:29 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:56:30.074 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:29 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:56:30.074 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:29 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:56:30.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:29 smithi110 ceph-mon[31709]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:56:30.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:29 smithi110 ceph-mon[31709]: osdmap e49: 7 total, 7 up, 7 in 2024-09-11T07:56:30.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:29 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:56:30.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:29 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:56:30.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:29 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:56:30.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:29 smithi110 ceph-mon[25392]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:56:30.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:29 smithi110 ceph-mon[25392]: osdmap e49: 7 total, 7 up, 7 in 2024-09-11T07:56:30.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:29 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:56:30.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:29 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:56:30.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:29 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:56:30.998 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:30 smithi139 ceph-mon[27311]: from='client.24259 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:56:30.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:30 smithi139 ceph-mon[27311]: osdmap e50: 7 total, 7 up, 7 in 2024-09-11T07:56:31.108 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:30 smithi110 ceph-mon[25392]: from='client.24259 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:56:31.109 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:30 smithi110 ceph-mon[25392]: osdmap e50: 7 total, 7 up, 7 in 2024-09-11T07:56:31.109 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:30 smithi110 ceph-mon[31709]: from='client.24259 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:56:31.109 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:30 smithi110 ceph-mon[31709]: osdmap e50: 7 total, 7 up, 7 in 2024-09-11T07:56:31.902 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:31 smithi139 ceph-mon[27311]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:56:31.902 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:31 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:31.903 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:31 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:31.903 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:31 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:32.052 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:31 smithi110 ceph-mon[31709]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:56:32.052 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:31 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:32.052 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:31 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:32.052 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:31 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:32.053 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:31 smithi110 ceph-mon[25392]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:56:32.053 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:31 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:32.053 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:31 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:32.053 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:31 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:32.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:32 smithi139 ceph-mon[27311]: from='client.? 172.21.15.139:0/861124502' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c58219c0-3caa-4b99-afb1-bc572ac152ed"}]: dispatch 2024-09-11T07:56:32.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:32 smithi139 ceph-mon[27311]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c58219c0-3caa-4b99-afb1-bc572ac152ed"}]: dispatch 2024-09-11T07:56:32.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:32 smithi139 ceph-mon[27311]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c58219c0-3caa-4b99-afb1-bc572ac152ed"}]': finished 2024-09-11T07:56:32.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:32 smithi139 ceph-mon[27311]: osdmap e51: 8 total, 7 up, 8 in 2024-09-11T07:56:32.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:32 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:56:32.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:32 smithi139 ceph-mon[27311]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 127 KiB/s, 0 objects/s recovering 2024-09-11T07:56:33.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:32 smithi110 ceph-mon[31709]: from='client.? 172.21.15.139:0/861124502' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c58219c0-3caa-4b99-afb1-bc572ac152ed"}]: dispatch 2024-09-11T07:56:33.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:32 smithi110 ceph-mon[31709]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c58219c0-3caa-4b99-afb1-bc572ac152ed"}]: dispatch 2024-09-11T07:56:33.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:32 smithi110 ceph-mon[31709]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c58219c0-3caa-4b99-afb1-bc572ac152ed"}]': finished 2024-09-11T07:56:33.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:32 smithi110 ceph-mon[31709]: osdmap e51: 8 total, 7 up, 8 in 2024-09-11T07:56:33.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:32 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:56:33.219 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:32 smithi110 ceph-mon[31709]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 127 KiB/s, 0 objects/s recovering 2024-09-11T07:56:33.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:32 smithi110 ceph-mon[25392]: from='client.? 172.21.15.139:0/861124502' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c58219c0-3caa-4b99-afb1-bc572ac152ed"}]: dispatch 2024-09-11T07:56:33.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:32 smithi110 ceph-mon[25392]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c58219c0-3caa-4b99-afb1-bc572ac152ed"}]: dispatch 2024-09-11T07:56:33.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:32 smithi110 ceph-mon[25392]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c58219c0-3caa-4b99-afb1-bc572ac152ed"}]': finished 2024-09-11T07:56:33.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:32 smithi110 ceph-mon[25392]: osdmap e51: 8 total, 7 up, 8 in 2024-09-11T07:56:33.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:32 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:56:33.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:32 smithi110 ceph-mon[25392]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 127 KiB/s, 0 objects/s recovering 2024-09-11T07:56:34.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:33 smithi110 ceph-mon[25392]: from='client.? 172.21.15.139:0/278848509' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:56:34.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:33 smithi110 ceph-mon[31709]: from='client.? 172.21.15.139:0/278848509' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:56:34.248 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:33 smithi139 ceph-mon[27311]: from='client.? 172.21.15.139:0/278848509' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:56:35.137 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:34 smithi139 ceph-mon[27311]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T07:56:35.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:34 smithi110 ceph-mon[25392]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T07:56:35.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:34 smithi110 ceph-mon[31709]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T07:56:37.589 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:37 smithi139 ceph-mon[27311]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-09-11T07:56:37.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:37 smithi110 ceph-mon[25392]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-09-11T07:56:37.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:37 smithi110 ceph-mon[31709]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-09-11T07:56:38.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:38 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-11T07:56:38.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:38 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:56:38.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:38 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-11T07:56:38.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:38 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:56:38.748 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:38 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-11T07:56:38.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:38 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:56:39.691 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:39 smithi139 ceph-mon[27311]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-11T07:56:39.691 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:39 smithi139 ceph-mon[27311]: Deploying daemon osd.7 on smithi139 2024-09-11T07:56:39.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:39 smithi110 ceph-mon[31709]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-11T07:56:39.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:39 smithi110 ceph-mon[31709]: Deploying daemon osd.7 on smithi139 2024-09-11T07:56:39.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:39 smithi110 ceph-mon[25392]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-11T07:56:39.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:39 smithi110 ceph-mon[25392]: Deploying daemon osd.7 on smithi139 2024-09-11T07:56:41.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:41 smithi139 ceph-mon[27311]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T07:56:41.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:41 smithi110 ceph-mon[25392]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T07:56:41.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:41 smithi110 ceph-mon[31709]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T07:56:43.498 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:43 smithi139 ceph-mon[27311]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:56:43.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:43 smithi110 ceph-mon[25392]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:56:43.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:43 smithi110 ceph-mon[31709]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:56:45.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:45 smithi139 ceph-mon[27311]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:56:45.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:45 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:56:45.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:45 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:45.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:45 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:56:45.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:45 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:56:45.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:45 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:45.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:45 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:45.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:45 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:45.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:45 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:45.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:45 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:45.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:45 smithi110 ceph-mon[25392]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:56:45.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:45 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:56:45.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:45 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:45.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:45 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:56:45.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:45 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:56:45.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:45 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:45.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:45 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:45.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:45 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:45.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:45 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:45.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:45 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:45.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:45 smithi110 ceph-mon[31709]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:56:45.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:45 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:56:45.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:45 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:45.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:45 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:56:45.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:45 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:56:45.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:45 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:45.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:45 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:45.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:45 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:45.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:45 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:45.720 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:45 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:47.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:47 smithi139 ceph-mon[27311]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:56:47.697 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:47 smithi110 ceph-mon[31709]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:56:47.697 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:47 smithi110 ceph-mon[25392]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:56:48.395 INFO:teuthology.orchestra.run.smithi139.stdout:Created osd(s) 7 on host 'smithi139' 2024-09-11T07:56:48.607 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:48 smithi110 ceph-mon[31709]: from='osd.7 [v2:172.21.15.139:6824/232507052,v1:172.21.15.139:6825/232507052]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T07:56:48.607 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:48 smithi110 ceph-mon[31709]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T07:56:48.607 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:48 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:56:48.607 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:48 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:48.608 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:48 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:56:48.608 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:48 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:56:48.608 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:48 smithi110 ceph-mon[25392]: from='osd.7 [v2:172.21.15.139:6824/232507052,v1:172.21.15.139:6825/232507052]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T07:56:48.608 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:48 smithi110 ceph-mon[25392]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T07:56:48.608 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:48 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:56:48.608 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:48 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:48.608 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:48 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:56:48.608 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:48 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:56:48.706 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:48 smithi139 ceph-mon[27311]: from='osd.7 [v2:172.21.15.139:6824/232507052,v1:172.21.15.139:6825/232507052]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T07:56:48.706 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:48 smithi139 ceph-mon[27311]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T07:56:48.706 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:48 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:56:48.706 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:48 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:48.706 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:48 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:56:48.706 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:48 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:56:49.023 DEBUG:teuthology.orchestra.run.smithi139:osd.7> sudo journalctl -f -n 0 -u ceph-41bd1096-7012-11ef-bcea-c7b262605968@osd.7.service 2024-09-11T07:56:49.026 INFO:tasks.cephadm:Waiting for 8 OSDs to come up... 2024-09-11T07:56:49.026 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-11T07:56:49.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:49 smithi110 ceph-mon[31709]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:56:49.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:49 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:49.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:49 smithi110 ceph-mon[31709]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-11T07:56:49.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:49 smithi110 ceph-mon[31709]: osdmap e52: 8 total, 7 up, 8 in 2024-09-11T07:56:49.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:49 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:56:49.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:49 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:56:49.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:49 smithi110 ceph-mon[31709]: from='osd.7 [v2:172.21.15.139:6824/232507052,v1:172.21.15.139:6825/232507052]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-09-11T07:56:49.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:49 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:56:49.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:49 smithi110 ceph-mon[31709]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-09-11T07:56:49.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:49 smithi110 ceph-mon[31709]: Adjusting osd_memory_target on smithi139 to 4016M 2024-09-11T07:56:49.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:49 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:49.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:49 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:56:49.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:49 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:56:49.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:49 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:49.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:49 smithi110 ceph-mon[25392]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:56:49.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:49 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:49.720 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:49 smithi110 ceph-mon[25392]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-11T07:56:49.720 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:49 smithi110 ceph-mon[25392]: osdmap e52: 8 total, 7 up, 8 in 2024-09-11T07:56:49.720 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:49 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:56:49.720 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:49 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:56:49.720 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:49 smithi110 ceph-mon[25392]: from='osd.7 [v2:172.21.15.139:6824/232507052,v1:172.21.15.139:6825/232507052]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-09-11T07:56:49.720 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:49 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:56:49.720 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:49 smithi110 ceph-mon[25392]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-09-11T07:56:49.720 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:49 smithi110 ceph-mon[25392]: Adjusting osd_memory_target on smithi139 to 4016M 2024-09-11T07:56:49.720 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:49 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:49.720 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:49 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:56:49.720 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:49 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:56:49.720 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:49 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:49.749 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 07:56:49 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T07:56:49.407+0000 7fdbad9b3640 -1 osd.7 0 waiting for initial osdmap 2024-09-11T07:56:49.749 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 07:56:49 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T07:56:49.415+0000 7fdba87ea640 -1 osd.7 53 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T07:56:49.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:49 smithi139 ceph-mon[27311]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:56:49.750 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:49 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:49.750 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:49 smithi139 ceph-mon[27311]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-11T07:56:49.750 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:49 smithi139 ceph-mon[27311]: osdmap e52: 8 total, 7 up, 8 in 2024-09-11T07:56:49.750 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:49 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:56:49.750 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:49 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:56:49.750 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:49 smithi139 ceph-mon[27311]: from='osd.7 [v2:172.21.15.139:6824/232507052,v1:172.21.15.139:6825/232507052]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-09-11T07:56:49.750 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:49 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:56:49.750 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:49 smithi139 ceph-mon[27311]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-09-11T07:56:49.750 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:49 smithi139 ceph-mon[27311]: Adjusting osd_memory_target on smithi139 to 4016M 2024-09-11T07:56:49.750 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:49 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:49.750 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:49 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:56:49.750 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:49 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:56:49.750 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:49 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:49.861 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:56:50.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:50 smithi110 ceph-mon[31709]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-09-11T07:56:50.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:50 smithi110 ceph-mon[31709]: osdmap e53: 8 total, 7 up, 8 in 2024-09-11T07:56:50.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:50 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:56:50.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:50 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:56:50.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:50 smithi110 ceph-mon[31709]: osd.7 [v2:172.21.15.139:6824/232507052,v1:172.21.15.139:6825/232507052] boot 2024-09-11T07:56:50.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:50 smithi110 ceph-mon[31709]: osdmap e54: 8 total, 8 up, 8 in 2024-09-11T07:56:50.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:50 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:56:50.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:50 smithi110 ceph-mon[25392]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-09-11T07:56:50.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:50 smithi110 ceph-mon[25392]: osdmap e53: 8 total, 7 up, 8 in 2024-09-11T07:56:50.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:50 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:56:50.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:50 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:56:50.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:50 smithi110 ceph-mon[25392]: osd.7 [v2:172.21.15.139:6824/232507052,v1:172.21.15.139:6825/232507052] boot 2024-09-11T07:56:50.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:50 smithi110 ceph-mon[25392]: osdmap e54: 8 total, 8 up, 8 in 2024-09-11T07:56:50.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:50 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:56:50.748 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:50 smithi139 ceph-mon[27311]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-09-11T07:56:50.748 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:50 smithi139 ceph-mon[27311]: osdmap e53: 8 total, 7 up, 8 in 2024-09-11T07:56:50.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:50 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:56:50.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:50 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:56:50.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:50 smithi139 ceph-mon[27311]: osd.7 [v2:172.21.15.139:6824/232507052,v1:172.21.15.139:6825/232507052] boot 2024-09-11T07:56:50.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:50 smithi139 ceph-mon[27311]: osdmap e54: 8 total, 8 up, 8 in 2024-09-11T07:56:50.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:50 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:56:51.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:51 smithi110 ceph-mon[25392]: purged_snaps scrub starts 2024-09-11T07:56:51.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:51 smithi110 ceph-mon[25392]: purged_snaps scrub ok 2024-09-11T07:56:51.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:51 smithi110 ceph-mon[25392]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:56:51.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:51 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:51.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:51 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:51.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:51 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:51.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:51 smithi110 ceph-mon[31709]: purged_snaps scrub starts 2024-09-11T07:56:51.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:51 smithi110 ceph-mon[31709]: purged_snaps scrub ok 2024-09-11T07:56:51.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:51 smithi110 ceph-mon[31709]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:56:51.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:51 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:51.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:51 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:51.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:51 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:51.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:51 smithi139 ceph-mon[27311]: purged_snaps scrub starts 2024-09-11T07:56:51.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:51 smithi139 ceph-mon[27311]: purged_snaps scrub ok 2024-09-11T07:56:51.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:51 smithi139 ceph-mon[27311]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:56:51.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:51 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:51.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:51 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:51.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:51 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:56:51.757 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T07:56:52.461 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":55,"num_osds":8,"num_up_osds":8,"osd_up_since":1726041410,"num_in_osds":8,"osd_in_since":1726041391,"num_remapped_pgs":1} 2024-09-11T07:56:52.462 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T07:56:52.701 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:56:52.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:52 smithi110 ceph-mon[25392]: osdmap e55: 8 total, 8 up, 8 in 2024-09-11T07:56:52.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:52 smithi110 ceph-mon[25392]: from='client.? 172.21.15.110:0/1428192529' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T07:56:52.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:52 smithi110 ceph-mon[31709]: osdmap e55: 8 total, 8 up, 8 in 2024-09-11T07:56:52.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:52 smithi110 ceph-mon[31709]: from='client.? 172.21.15.110:0/1428192529' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T07:56:52.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:52 smithi139 ceph-mon[27311]: osdmap e55: 8 total, 8 up, 8 in 2024-09-11T07:56:52.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:52 smithi139 ceph-mon[27311]: from='client.? 172.21.15.110:0/1428192529' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T07:56:53.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:53 smithi110 ceph-mon[31709]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:56:53.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:53 smithi110 ceph-mon[31709]: osdmap e56: 8 total, 8 up, 8 in 2024-09-11T07:56:53.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:53 smithi110 ceph-mon[25392]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:56:53.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:53 smithi110 ceph-mon[25392]: osdmap e56: 8 total, 8 up, 8 in 2024-09-11T07:56:53.748 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:53 smithi139 ceph-mon[27311]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:56:53.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:53 smithi139 ceph-mon[27311]: osdmap e56: 8 total, 8 up, 8 in 2024-09-11T07:56:54.558 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T07:56:54.558 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":58,"fsid":"41bd1096-7012-11ef-bcea-c7b262605968","created":"2024-09-11T07:51:48.770375+0000","modified":"2024-09-11T07:56:54.429986+0000","last_up_change":"2024-09-11T07:56:50.396037+0000","last_in_change":"2024-09-11T07:56:31.845597+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T07:54:50.418595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"05f2cdd1-77dc-4b26-afa8-16099c528c10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":2716989082},{"type":"v1","addr":"172.21.15.110:6803","nonce":2716989082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":2716989082},{"type":"v1","addr":"172.21.15.110:6805","nonce":2716989082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":2716989082},{"type":"v1","addr":"172.21.15.110:6809","nonce":2716989082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":2716989082},{"type":"v1","addr":"172.21.15.110:6807","nonce":2716989082}]},"public_addr":"172.21.15.110:6803/2716989082","cluster_addr":"172.21.15.110:6805/2716989082","heartbeat_back_addr":"172.21.15.110:6809/2716989082","heartbeat_front_addr":"172.21.15.110:6807/2716989082","state":["exists","up"]},{"osd":1,"uuid":"1e01c286-b92c-46b3-b61c-f8aea652b8be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":1928520079},{"type":"v1","addr":"172.21.15.110:6811","nonce":1928520079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":1928520079},{"type":"v1","addr":"172.21.15.110:6813","nonce":1928520079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":1928520079},{"type":"v1","addr":"172.21.15.110:6817","nonce":1928520079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":1928520079},{"type":"v1","addr":"172.21.15.110:6815","nonce":1928520079}]},"public_addr":"172.21.15.110:6811/1928520079","cluster_addr":"172.21.15.110:6813/1928520079","heartbeat_back_addr":"172.21.15.110:6817/1928520079","heartbeat_front_addr":"172.21.15.110:6815/1928520079","state":["exists","up"]},{"osd":2,"uuid":"b413fbb5-f09d-4a04-a72d-1cc5d554f90d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":3894315382},{"type":"v1","addr":"172.21.15.110:6819","nonce":3894315382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":3894315382},{"type":"v1","addr":"172.21.15.110:6821","nonce":3894315382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":3894315382},{"type":"v1","addr":"172.21.15.110:6825","nonce":3894315382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":3894315382},{"type":"v1","addr":"172.21.15.110:6823","nonce":3894315382}]},"public_addr":"172.21.15.110:6819/3894315382","cluster_addr":"172.21.15.110:6821/3894315382","heartbeat_back_addr":"172.21.15.110:6825/3894315382","heartbeat_front_addr":"172.21.15.110:6823/3894315382","state":["exists","up"]},{"osd":3,"uuid":"293d5b4d-11ef-4d6a-9b65-5a6b47472c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":1640229305},{"type":"v1","addr":"172.21.15.110:6827","nonce":1640229305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":1640229305},{"type":"v1","addr":"172.21.15.110:6829","nonce":1640229305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":1640229305},{"type":"v1","addr":"172.21.15.110:6833","nonce":1640229305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":1640229305},{"type":"v1","addr":"172.21.15.110:6831","nonce":1640229305}]},"public_addr":"172.21.15.110:6827/1640229305","cluster_addr":"172.21.15.110:6829/1640229305","heartbeat_back_addr":"172.21.15.110:6833/1640229305","heartbeat_front_addr":"172.21.15.110:6831/1640229305","state":["exists","up"]},{"osd":4,"uuid":"f9119e96-9863-45b7-b2d5-89bc3eeb1fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2399332353},{"type":"v1","addr":"172.21.15.139:6801","nonce":2399332353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2399332353},{"type":"v1","addr":"172.21.15.139:6803","nonce":2399332353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2399332353},{"type":"v1","addr":"172.21.15.139:6807","nonce":2399332353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2399332353},{"type":"v1","addr":"172.21.15.139:6805","nonce":2399332353}]},"public_addr":"172.21.15.139:6801/2399332353","cluster_addr":"172.21.15.139:6803/2399332353","heartbeat_back_addr":"172.21.15.139:6807/2399332353","heartbeat_front_addr":"172.21.15.139:6805/2399332353","state":["exists","up"]},{"osd":5,"uuid":"3339eb92-7017-4bb8-947f-557a32723e94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1641418305},{"type":"v1","addr":"172.21.15.139:6809","nonce":1641418305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1641418305},{"type":"v1","addr":"172.21.15.139:6811","nonce":1641418305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1641418305},{"type":"v1","addr":"172.21.15.139:6815","nonce":1641418305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1641418305},{"type":"v1","addr":"172.21.15.139:6813","nonce":1641418305}]},"public_addr":"172.21.15.139:6809/1641418305","cluster_addr":"172.21.15.139:6811/1641418305","heartbeat_back_addr":"172.21.15.139:6815/1641418305","heartbeat_front_addr":"172.21.15.139:6813/1641418305","state":["exists","up"]},{"osd":6,"uuid":"69a4457c-39cc-41f5-bceb-9229b8b030f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1783109268},{"type":"v1","addr":"172.21.15.139:6817","nonce":1783109268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1783109268},{"type":"v1","addr":"172.21.15.139:6819","nonce":1783109268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1783109268},{"type":"v1","addr":"172.21.15.139:6823","nonce":1783109268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1783109268},{"type":"v1","addr":"172.21.15.139:6821","nonce":1783109268}]},"public_addr":"172.21.15.139:6817/1783109268","cluster_addr":"172.21.15.139:6819/1783109268","heartbeat_back_addr":"172.21.15.139:6823/1783109268","heartbeat_front_addr":"172.21.15.139:6821/1783109268","state":["exists","up"]},{"osd":7,"uuid":"c58219c0-3caa-4b99-afb1-bc572ac152ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":232507052},{"type":"v1","addr":"172.21.15.139:6825","nonce":232507052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":232507052},{"type":"v1","addr":"172.21.15.139:6827","nonce":232507052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":232507052},{"type":"v1","addr":"172.21.15.139:6831","nonce":232507052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":232507052},{"type":"v1","addr":"172.21.15.139:6829","nonce":232507052}]},"public_addr":"172.21.15.139:6825/232507052","cluster_addr":"172.21.15.139:6827/232507052","heartbeat_back_addr":"172.21.15.139:6831/232507052","heartbeat_front_addr":"172.21.15.139:6829/232507052","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:53:57.309408+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:54:22.407580+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:54:47.670630+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:55:13.231294+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:55:37.035454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:56:00.079531+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:56:24.721942+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:56:48.981334+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.110:6801/4096534802":"2024-09-12T07:52:32.111455+0000","172.21.15.110:0/2391676173":"2024-09-12T07:52:32.111455+0000","172.21.15.110:0/1603882420":"2024-09-12T07:52:32.111455+0000","172.21.15.110:6800/4096534802":"2024-09-12T07:52:32.111455+0000","172.21.15.110:0/2727007179":"2024-09-12T07:52:10.347528+0000","172.21.15.110:0/1589810572":"2024-09-12T07:52:10.347528+0000","172.21.15.110:0/828045681":"2024-09-12T07:52:10.347528+0000","172.21.15.110:6801/1242491477":"2024-09-12T07:52:10.347528+0000","172.21.15.110:0/3707065533":"2024-09-12T07:52:32.111455+0000","172.21.15.110:6800/1242491477":"2024-09-12T07:52:10.347528+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T07:56:54.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:54 smithi110 ceph-mon[25392]: osdmap e57: 8 total, 8 up, 8 in 2024-09-11T07:56:54.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:54 smithi110 ceph-mon[31709]: osdmap e57: 8 total, 8 up, 8 in 2024-09-11T07:56:54.748 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:54 smithi139 ceph-mon[27311]: osdmap e57: 8 total, 8 up, 8 in 2024-09-11T07:56:55.209 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T07:54:50.418595+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-09-11T07:56:55.209 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-11T07:56:55.449 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:56:55.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:55 smithi110 ceph-mon[31709]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:56:55.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:55 smithi110 ceph-mon[31709]: osdmap e58: 8 total, 8 up, 8 in 2024-09-11T07:56:55.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:55 smithi110 ceph-mon[31709]: from='client.? 172.21.15.110:0/4185550218' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T07:56:55.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:55 smithi110 ceph-mon[25392]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:56:55.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:55 smithi110 ceph-mon[25392]: osdmap e58: 8 total, 8 up, 8 in 2024-09-11T07:56:55.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:55 smithi110 ceph-mon[25392]: from='client.? 172.21.15.110:0/4185550218' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T07:56:55.748 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:55 smithi139 ceph-mon[27311]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:56:55.748 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:55 smithi139 ceph-mon[27311]: osdmap e58: 8 total, 8 up, 8 in 2024-09-11T07:56:55.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:55 smithi139 ceph-mon[27311]: from='client.? 172.21.15.110:0/4185550218' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T07:56:57.349 INFO:teuthology.orchestra.run.smithi110.stdout:pg_num: 1 2024-09-11T07:56:57.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:57 smithi110 ceph-mon[25392]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-09-11T07:56:57.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:57 smithi110 ceph-mon[25392]: from='client.? 172.21.15.110:0/1097718220' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T07:56:57.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:57 smithi110 ceph-mon[31709]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-09-11T07:56:57.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:57 smithi110 ceph-mon[31709]: from='client.? 172.21.15.110:0/1097718220' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T07:56:57.748 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:57 smithi139 ceph-mon[27311]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-09-11T07:56:57.748 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:57 smithi139 ceph-mon[27311]: from='client.? 172.21.15.110:0/1097718220' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T07:56:58.057 INFO:tasks.cephadm:Adding ceph.rgw.foo.a on smithi110 2024-09-11T07:56:58.057 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- ceph orch apply rgw foo.a --placement '1;smithi110=foo.a' 2024-09-11T07:56:58.307 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:56:59.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:56:59 smithi110 ceph-mon[25392]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T07:56:59.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:56:59 smithi110 ceph-mon[31709]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T07:56:59.748 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:56:59 smithi139 ceph-mon[27311]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T07:57:00.150 INFO:teuthology.orchestra.run.smithi139.stdout:Scheduled rgw.foo.a update... 2024-09-11T07:57:00.687 DEBUG:teuthology.orchestra.run.smithi110:rgw.foo.a> sudo journalctl -f -n 0 -u ceph-41bd1096-7012-11ef-bcea-c7b262605968@rgw.foo.a.service 2024-09-11T07:57:00.690 INFO:tasks.cephadm:Adding ceph.iscsi.iscsi.a on smithi139 2024-09-11T07:57:00.690 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- ceph osd pool create datapool 3 3 replicated 2024-09-11T07:57:00.935 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:57:01.348 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:01 smithi139 ceph-mon[27311]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi110=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:57:01.348 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:01 smithi139 ceph-mon[27311]: Saving service rgw.foo.a spec with placement smithi110=foo.a;count:1 2024-09-11T07:57:01.348 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:01 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:01.348 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:01 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:57:01.348 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:01 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:01.348 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:01 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:57:01.349 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:01 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:01.349 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:01 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-11T07:57:01.349 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:01 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-11T07:57:01.349 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:01 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:01.349 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:01 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:01.349 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:01 smithi139 ceph-mon[27311]: Deploying daemon rgw.foo.a on smithi110 2024-09-11T07:57:01.349 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:01 smithi139 ceph-mon[27311]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-11T07:57:01.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:01 smithi110 ceph-mon[25392]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi110=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:57:01.469 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:01 smithi110 ceph-mon[25392]: Saving service rgw.foo.a spec with placement smithi110=foo.a;count:1 2024-09-11T07:57:01.469 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:01 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:01.469 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:01 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:57:01.469 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:01 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:01.469 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:01 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:57:01.469 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:01 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:01.469 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:01 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-11T07:57:01.469 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:01 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-11T07:57:01.469 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:01 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:01.469 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:01 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:01.469 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:01 smithi110 ceph-mon[25392]: Deploying daemon rgw.foo.a on smithi110 2024-09-11T07:57:01.469 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:01 smithi110 ceph-mon[25392]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-11T07:57:01.470 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:01 smithi110 ceph-mon[31709]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi110=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:57:01.470 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:01 smithi110 ceph-mon[31709]: Saving service rgw.foo.a spec with placement smithi110=foo.a;count:1 2024-09-11T07:57:01.470 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:01 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:01.470 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:01 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:57:01.470 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:01 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:01.470 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:01 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:57:01.470 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:01 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:01.470 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:01 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-11T07:57:01.470 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:01 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-11T07:57:01.470 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:01 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:01.470 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:01 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:01.470 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:01 smithi110 ceph-mon[31709]: Deploying daemon rgw.foo.a on smithi110 2024-09-11T07:57:01.470 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:01 smithi110 ceph-mon[31709]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-11T07:57:02.218 INFO:journalctl@ceph.rgw.foo.a.smithi110.stdout:Sep 11 07:57:02 smithi110 systemd[1]: Starting Ceph rgw.foo.a for 41bd1096-7012-11ef-bcea-c7b262605968... 2024-09-11T07:57:02.567 INFO:journalctl@ceph.rgw.foo.a.smithi110.stdout:Sep 11 07:57:02 smithi110 podman[50332]: 2024-09-11 07:57:02.15198616 +0000 UTC m=+0.016090920 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T07:57:02.567 INFO:journalctl@ceph.rgw.foo.a.smithi110.stdout:Sep 11 07:57:02 smithi110 podman[50332]: 2024-09-11 07:57:02.255774381 +0000 UTC m=+0.119879125 container create 3cb6994aa1dd473e78c075de2f846aa74f226a7ae09293a701bff5edf00d3929 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-rgw-foo-a, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-11T07:57:02.567 INFO:journalctl@ceph.rgw.foo.a.smithi110.stdout:Sep 11 07:57:02 smithi110 podman[50332]: 2024-09-11 07:57:02.483549631 +0000 UTC m=+0.347654381 container init 3cb6994aa1dd473e78c075de2f846aa74f226a7ae09293a701bff5edf00d3929 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-rgw-foo-a, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-11T07:57:02.567 INFO:journalctl@ceph.rgw.foo.a.smithi110.stdout:Sep 11 07:57:02 smithi110 podman[50332]: 2024-09-11 07:57:02.486910655 +0000 UTC m=+0.351015399 container start 3cb6994aa1dd473e78c075de2f846aa74f226a7ae09293a701bff5edf00d3929 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-rgw-foo-a, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-11T07:57:02.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:02 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:02.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:02 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:02.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:02 smithi110 ceph-mon[31709]: Detected new or changed devices on smithi139 2024-09-11T07:57:02.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:02 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:02.568 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:02 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:02.568 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:02 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:02.568 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:02 smithi110 ceph-mon[25392]: Detected new or changed devices on smithi139 2024-09-11T07:57:02.569 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:02 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:02.706 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:02 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:02.706 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:02 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:02.706 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:02 smithi139 ceph-mon[27311]: Detected new or changed devices on smithi139 2024-09-11T07:57:02.706 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:02 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:02.968 INFO:journalctl@ceph.rgw.foo.a.smithi110.stdout:Sep 11 07:57:02 smithi110 bash[50332]: 3cb6994aa1dd473e78c075de2f846aa74f226a7ae09293a701bff5edf00d3929 2024-09-11T07:57:02.968 INFO:journalctl@ceph.rgw.foo.a.smithi110.stdout:Sep 11 07:57:02 smithi110 systemd[1]: Started Ceph rgw.foo.a for 41bd1096-7012-11ef-bcea-c7b262605968. 2024-09-11T07:57:03.476 INFO:teuthology.orchestra.run.smithi139.stderr:pool 'datapool' created 2024-09-11T07:57:03.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:03 smithi110 ceph-mon[31709]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-11T07:57:03.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:03 smithi110 ceph-mon[31709]: from='client.? 172.21.15.139:0/2212490136' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-11T07:57:03.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:03 smithi110 ceph-mon[31709]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-11T07:57:03.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:03 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:03.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:03 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:03.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:03 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:03.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:03 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:03.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:03 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:03.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:03 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:57:03.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:03 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:03.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:03 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:57:03.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:03 smithi110 ceph-mon[25392]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-11T07:57:03.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:03 smithi110 ceph-mon[25392]: from='client.? 172.21.15.139:0/2212490136' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-11T07:57:03.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:03 smithi110 ceph-mon[25392]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-11T07:57:03.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:03 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:03.720 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:03 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:03.720 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:03 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:03.720 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:03 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:03.720 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:03 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:03.720 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:03 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:57:03.720 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:03 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:03.720 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:03 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:57:03.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:03 smithi139 ceph-mon[27311]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-11T07:57:03.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:03 smithi139 ceph-mon[27311]: from='client.? 172.21.15.139:0/2212490136' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-11T07:57:03.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:03 smithi139 ceph-mon[27311]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-11T07:57:03.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:03 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:03.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:03 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:03.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:03 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:03.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:03 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:03.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:03 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:03.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:03 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:57:03.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:03 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:03.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:03 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:57:04.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:04 smithi139 ceph-mon[27311]: Saving service rgw.foo.a spec with placement smithi110=foo.a;count:1 2024-09-11T07:57:04.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:04 smithi139 ceph-mon[27311]: Checking dashboard <-> RGW credentials 2024-09-11T07:57:04.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:04 smithi139 ceph-mon[27311]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-11T07:57:04.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:04 smithi139 ceph-mon[27311]: osdmap e59: 8 total, 8 up, 8 in 2024-09-11T07:57:04.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:04 smithi139 ceph-mon[27311]: from='client.? 172.21.15.110:0/3139905262' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-11T07:57:04.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:04 smithi139 ceph-mon[27311]: from='client.? 172.21.15.110:0/1038013311' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-11T07:57:04.606 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:04 smithi110 ceph-mon[31709]: Saving service rgw.foo.a spec with placement smithi110=foo.a;count:1 2024-09-11T07:57:04.606 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:04 smithi110 ceph-mon[31709]: Checking dashboard <-> RGW credentials 2024-09-11T07:57:04.606 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:04 smithi110 ceph-mon[31709]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-11T07:57:04.606 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:04 smithi110 ceph-mon[31709]: osdmap e59: 8 total, 8 up, 8 in 2024-09-11T07:57:04.606 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:04 smithi110 ceph-mon[31709]: from='client.? 172.21.15.110:0/3139905262' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-11T07:57:04.606 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:04 smithi110 ceph-mon[31709]: from='client.? 172.21.15.110:0/1038013311' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-11T07:57:04.607 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:04 smithi110 ceph-mon[25392]: Saving service rgw.foo.a spec with placement smithi110=foo.a;count:1 2024-09-11T07:57:04.607 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:04 smithi110 ceph-mon[25392]: Checking dashboard <-> RGW credentials 2024-09-11T07:57:04.607 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:04 smithi110 ceph-mon[25392]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-11T07:57:04.607 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:04 smithi110 ceph-mon[25392]: osdmap e59: 8 total, 8 up, 8 in 2024-09-11T07:57:04.607 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:04 smithi110 ceph-mon[25392]: from='client.? 172.21.15.110:0/3139905262' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-11T07:57:04.607 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:04 smithi110 ceph-mon[25392]: from='client.? 172.21.15.110:0/1038013311' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-11T07:57:04.744 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- rbd pool init datapool 2024-09-11T07:57:05.537 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:57:05.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:05 smithi110 ceph-mon[31709]: pgmap v189: 36 pgs: 11 creating+peering, 24 unknown, 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T07:57:05.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:05 smithi110 ceph-mon[31709]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:57:05.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:05 smithi110 ceph-mon[31709]: from='client.? 172.21.15.110:0/3139905262' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-11T07:57:05.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:05 smithi110 ceph-mon[31709]: from='client.? 172.21.15.110:0/1038013311' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-11T07:57:05.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:05 smithi110 ceph-mon[31709]: osdmap e60: 8 total, 8 up, 8 in 2024-09-11T07:57:05.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:05 smithi110 ceph-mon[25392]: pgmap v189: 36 pgs: 11 creating+peering, 24 unknown, 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T07:57:05.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:05 smithi110 ceph-mon[25392]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:57:05.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:05 smithi110 ceph-mon[25392]: from='client.? 172.21.15.110:0/3139905262' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-11T07:57:05.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:05 smithi110 ceph-mon[25392]: from='client.? 172.21.15.110:0/1038013311' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-11T07:57:05.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:05 smithi110 ceph-mon[25392]: osdmap e60: 8 total, 8 up, 8 in 2024-09-11T07:57:05.733 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:05 smithi139 ceph-mon[27311]: pgmap v189: 36 pgs: 11 creating+peering, 24 unknown, 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T07:57:05.734 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:05 smithi139 ceph-mon[27311]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:57:05.734 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:05 smithi139 ceph-mon[27311]: from='client.? 172.21.15.110:0/3139905262' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-11T07:57:05.734 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:05 smithi139 ceph-mon[27311]: from='client.? 172.21.15.110:0/1038013311' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-11T07:57:05.734 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:05 smithi139 ceph-mon[27311]: osdmap e60: 8 total, 8 up, 8 in 2024-09-11T07:57:06.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:06 smithi139 ceph-mon[27311]: osdmap e61: 8 total, 8 up, 8 in 2024-09-11T07:57:06.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:06 smithi139 ceph-mon[27311]: from='client.? 172.21.15.110:0/3139905262' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T07:57:06.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:06 smithi139 ceph-mon[27311]: from='client.? 172.21.15.110:0/1372091791' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T07:57:06.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:06 smithi139 ceph-mon[27311]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T07:57:06.953 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:06 smithi110 ceph-mon[25392]: osdmap e61: 8 total, 8 up, 8 in 2024-09-11T07:57:06.953 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:06 smithi110 ceph-mon[25392]: from='client.? 172.21.15.110:0/3139905262' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T07:57:06.953 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:06 smithi110 ceph-mon[25392]: from='client.? 172.21.15.110:0/1372091791' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T07:57:06.953 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:06 smithi110 ceph-mon[25392]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T07:57:06.953 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:06 smithi110 ceph-mon[31709]: osdmap e61: 8 total, 8 up, 8 in 2024-09-11T07:57:06.953 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:06 smithi110 ceph-mon[31709]: from='client.? 172.21.15.110:0/3139905262' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T07:57:06.953 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:06 smithi110 ceph-mon[31709]: from='client.? 172.21.15.110:0/1372091791' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T07:57:06.953 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:06 smithi110 ceph-mon[31709]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T07:57:07.748 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:07 smithi139 ceph-mon[27311]: pgmap v192: 68 pgs: 15 active+clean, 19 creating+peering, 34 unknown; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 170 B/s wr, 0 op/s 2024-09-11T07:57:07.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:07 smithi139 ceph-mon[27311]: from='client.? 172.21.15.110:0/3139905262' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-11T07:57:07.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:07 smithi139 ceph-mon[27311]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-11T07:57:07.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:07 smithi139 ceph-mon[27311]: osdmap e62: 8 total, 8 up, 8 in 2024-09-11T07:57:07.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:07 smithi139 ceph-mon[27311]: from='client.? 172.21.15.139:0/2206074423' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-11T07:57:07.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:07 smithi139 ceph-mon[27311]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-11T07:57:07.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:07 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:07.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:07 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:07.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:07 smithi110 ceph-mon[31709]: pgmap v192: 68 pgs: 15 active+clean, 19 creating+peering, 34 unknown; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 170 B/s wr, 0 op/s 2024-09-11T07:57:07.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:07 smithi110 ceph-mon[31709]: from='client.? 172.21.15.110:0/3139905262' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-11T07:57:07.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:07 smithi110 ceph-mon[31709]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-11T07:57:07.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:07 smithi110 ceph-mon[31709]: osdmap e62: 8 total, 8 up, 8 in 2024-09-11T07:57:07.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:07 smithi110 ceph-mon[31709]: from='client.? 172.21.15.139:0/2206074423' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-11T07:57:07.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:07 smithi110 ceph-mon[31709]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-11T07:57:07.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:07 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:07.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:07 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:07.970 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:07 smithi110 ceph-mon[25392]: pgmap v192: 68 pgs: 15 active+clean, 19 creating+peering, 34 unknown; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 170 B/s wr, 0 op/s 2024-09-11T07:57:07.970 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:07 smithi110 ceph-mon[25392]: from='client.? 172.21.15.110:0/3139905262' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-11T07:57:07.970 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:07 smithi110 ceph-mon[25392]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-11T07:57:07.970 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:07 smithi110 ceph-mon[25392]: osdmap e62: 8 total, 8 up, 8 in 2024-09-11T07:57:07.970 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:07 smithi110 ceph-mon[25392]: from='client.? 172.21.15.139:0/2206074423' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-11T07:57:07.971 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:07 smithi110 ceph-mon[25392]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-11T07:57:07.971 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:07 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:07.971 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:07 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:08.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:08 smithi110 ceph-mon[31709]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-11T07:57:08.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:08 smithi110 ceph-mon[31709]: osdmap e63: 8 total, 8 up, 8 in 2024-09-11T07:57:08.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:08 smithi110 ceph-mon[31709]: from='client.? 172.21.15.110:0/1372091791' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T07:57:08.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:08 smithi110 ceph-mon[31709]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T07:57:08.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:08 smithi110 ceph-mon[31709]: from='client.? 172.21.15.110:0/3139905262' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T07:57:08.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:08 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:08.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:08 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:08.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:08 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:08.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:08 smithi110 ceph-mon[25392]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-11T07:57:08.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:08 smithi110 ceph-mon[25392]: osdmap e63: 8 total, 8 up, 8 in 2024-09-11T07:57:08.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:08 smithi110 ceph-mon[25392]: from='client.? 172.21.15.110:0/1372091791' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T07:57:08.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:08 smithi110 ceph-mon[25392]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T07:57:08.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:08 smithi110 ceph-mon[25392]: from='client.? 172.21.15.110:0/3139905262' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T07:57:08.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:08 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:08.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:08 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:08.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:08 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:08.998 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:08 smithi139 ceph-mon[27311]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-11T07:57:08.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:08 smithi139 ceph-mon[27311]: osdmap e63: 8 total, 8 up, 8 in 2024-09-11T07:57:08.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:08 smithi139 ceph-mon[27311]: from='client.? 172.21.15.110:0/1372091791' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T07:57:08.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:08 smithi139 ceph-mon[27311]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T07:57:08.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:08 smithi139 ceph-mon[27311]: from='client.? 172.21.15.110:0/3139905262' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T07:57:08.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:08 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:08.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:08 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:08.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:08 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:09.768 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:09 smithi139 ceph-mon[27311]: pgmap v195: 100 pgs: 30 active+clean, 24 creating+peering, 46 unknown; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T07:57:09.768 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:09 smithi139 ceph-mon[27311]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-11T07:57:09.768 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:09 smithi139 ceph-mon[27311]: from='client.? 172.21.15.110:0/3139905262' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-11T07:57:09.768 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:09 smithi139 ceph-mon[27311]: osdmap e64: 8 total, 8 up, 8 in 2024-09-11T07:57:09.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:09 smithi110 ceph-mon[25392]: pgmap v195: 100 pgs: 30 active+clean, 24 creating+peering, 46 unknown; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T07:57:09.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:09 smithi110 ceph-mon[25392]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-11T07:57:09.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:09 smithi110 ceph-mon[25392]: from='client.? 172.21.15.110:0/3139905262' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-11T07:57:09.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:09 smithi110 ceph-mon[25392]: osdmap e64: 8 total, 8 up, 8 in 2024-09-11T07:57:09.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:09 smithi110 ceph-mon[31709]: pgmap v195: 100 pgs: 30 active+clean, 24 creating+peering, 46 unknown; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T07:57:09.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:09 smithi110 ceph-mon[31709]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-11T07:57:09.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:09 smithi110 ceph-mon[31709]: from='client.? 172.21.15.110:0/3139905262' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-11T07:57:09.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:09 smithi110 ceph-mon[31709]: osdmap e64: 8 total, 8 up, 8 in 2024-09-11T07:57:10.015 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- ceph orch apply iscsi datapool admin admin --trusted_ip_list 172.21.15.139 --placement '1;smithi139=iscsi.a' 2024-09-11T07:57:10.261 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:57:10.671 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:10 smithi139 ceph-mon[27311]: osdmap e65: 8 total, 8 up, 8 in 2024-09-11T07:57:10.671 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:10 smithi139 ceph-mon[27311]: from='client.? 172.21.15.110:0/1475963597' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T07:57:10.671 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:10 smithi139 ceph-mon[27311]: from='client.? 172.21.15.110:0/3889215479' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T07:57:10.672 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:10 smithi139 ceph-mon[27311]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T07:57:10.672 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:10 smithi139 ceph-mon[27311]: from='client.? 172.21.15.110:0/1475963597' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-11T07:57:10.672 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:10 smithi139 ceph-mon[27311]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-11T07:57:10.672 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:10 smithi139 ceph-mon[27311]: osdmap e66: 8 total, 8 up, 8 in 2024-09-11T07:57:10.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:10 smithi110 ceph-mon[25392]: osdmap e65: 8 total, 8 up, 8 in 2024-09-11T07:57:10.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:10 smithi110 ceph-mon[25392]: from='client.? 172.21.15.110:0/1475963597' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T07:57:10.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:10 smithi110 ceph-mon[25392]: from='client.? 172.21.15.110:0/3889215479' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T07:57:10.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:10 smithi110 ceph-mon[25392]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T07:57:10.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:10 smithi110 ceph-mon[25392]: from='client.? 172.21.15.110:0/1475963597' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-11T07:57:10.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:10 smithi110 ceph-mon[25392]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-11T07:57:10.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:10 smithi110 ceph-mon[25392]: osdmap e66: 8 total, 8 up, 8 in 2024-09-11T07:57:10.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:10 smithi110 ceph-mon[31709]: osdmap e65: 8 total, 8 up, 8 in 2024-09-11T07:57:10.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:10 smithi110 ceph-mon[31709]: from='client.? 172.21.15.110:0/1475963597' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T07:57:10.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:10 smithi110 ceph-mon[31709]: from='client.? 172.21.15.110:0/3889215479' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T07:57:10.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:10 smithi110 ceph-mon[31709]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T07:57:10.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:10 smithi110 ceph-mon[31709]: from='client.? 172.21.15.110:0/1475963597' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-11T07:57:10.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:10 smithi110 ceph-mon[31709]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-11T07:57:10.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:10 smithi110 ceph-mon[31709]: osdmap e66: 8 total, 8 up, 8 in 2024-09-11T07:57:11.622 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:11 smithi139 ceph-mon[27311]: pgmap v198: 132 pgs: 71 active+clean, 13 creating+peering, 48 unknown; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.0 KiB/s rd, 1023 B/s wr, 5 op/s 2024-09-11T07:57:11.622 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:11 smithi139 ceph-mon[27311]: from='client.? 172.21.15.110:0/1475963597' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T07:57:11.622 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:11 smithi139 ceph-mon[27311]: from='client.? 172.21.15.110:0/3889215479' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T07:57:11.622 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:11 smithi139 ceph-mon[27311]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T07:57:11.622 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:11 smithi139 ceph-mon[27311]: from='client.? 172.21.15.110:0/1475963597' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-11T07:57:11.622 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:11 smithi139 ceph-mon[27311]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-11T07:57:11.622 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:11 smithi139 ceph-mon[27311]: osdmap e67: 8 total, 8 up, 8 in 2024-09-11T07:57:11.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:11 smithi110 ceph-mon[31709]: pgmap v198: 132 pgs: 71 active+clean, 13 creating+peering, 48 unknown; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.0 KiB/s rd, 1023 B/s wr, 5 op/s 2024-09-11T07:57:11.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:11 smithi110 ceph-mon[31709]: from='client.? 172.21.15.110:0/1475963597' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T07:57:11.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:11 smithi110 ceph-mon[31709]: from='client.? 172.21.15.110:0/3889215479' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T07:57:11.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:11 smithi110 ceph-mon[31709]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T07:57:11.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:11 smithi110 ceph-mon[31709]: from='client.? 172.21.15.110:0/1475963597' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-11T07:57:11.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:11 smithi110 ceph-mon[31709]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-11T07:57:11.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:11 smithi110 ceph-mon[31709]: osdmap e67: 8 total, 8 up, 8 in 2024-09-11T07:57:11.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:11 smithi110 ceph-mon[25392]: pgmap v198: 132 pgs: 71 active+clean, 13 creating+peering, 48 unknown; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.0 KiB/s rd, 1023 B/s wr, 5 op/s 2024-09-11T07:57:11.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:11 smithi110 ceph-mon[25392]: from='client.? 172.21.15.110:0/1475963597' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T07:57:11.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:11 smithi110 ceph-mon[25392]: from='client.? 172.21.15.110:0/3889215479' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T07:57:11.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:11 smithi110 ceph-mon[25392]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T07:57:11.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:11 smithi110 ceph-mon[25392]: from='client.? 172.21.15.110:0/1475963597' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-11T07:57:11.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:11 smithi110 ceph-mon[25392]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-11T07:57:11.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:11 smithi110 ceph-mon[25392]: osdmap e67: 8 total, 8 up, 8 in 2024-09-11T07:57:12.065 INFO:teuthology.orchestra.run.smithi139.stdout:Scheduled iscsi.datapool update... 2024-09-11T07:57:12.689 INFO:tasks.cephadm:Distributing iscsi-gateway.cfg... 2024-09-11T07:57:12.689 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-11T07:57:12.689 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-09-11T07:57:12.720 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-11T07:57:12.720 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-09-11T07:57:12.748 DEBUG:teuthology.orchestra.run.smithi139:iscsi.iscsi.a> sudo journalctl -f -n 0 -u ceph-41bd1096-7012-11ef-bcea-c7b262605968@iscsi.iscsi.a.service 2024-09-11T07:57:12.791 INFO:tasks.cephadm:Adding prometheus.a on smithi139 2024-09-11T07:57:12.792 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- ceph orch apply prometheus '1;smithi139=a' 2024-09-11T07:57:12.918 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:12 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:12.919 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:12 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:12.919 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:12 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:12.919 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:12 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:57:12.919 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:12 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:12.919 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:12 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:57:12.919 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:12 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:12.919 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:12 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:12.919 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:12 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:57:12.919 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:12 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:12.919 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:12 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:57:12.919 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:12 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:12.919 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:12 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-11T07:57:12.920 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:12 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-11T07:57:12.920 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:12 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:12.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:12 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:12.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:12 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:12.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:12 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:12.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:12 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:57:12.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:12 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:12.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:12 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:57:12.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:12 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:12.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:12 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:12.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:12 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:57:12.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:12 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:12.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:12 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:57:12.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:12 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:12.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:12 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-11T07:57:12.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:12 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-11T07:57:12.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:12 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:12.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:12 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:12.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:12 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:12.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:12 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:12.970 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:12 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:57:12.970 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:12 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:12.970 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:12 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:57:12.970 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:12 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:12.970 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:12 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:12.970 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:12 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:57:12.970 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:12 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:12.970 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:12 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:57:12.970 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:12 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:12.970 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:12 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-11T07:57:12.970 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:12 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-11T07:57:12.970 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:12 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:13.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:13 smithi110 ceph-mon[25392]: Checking dashboard <-> RGW credentials 2024-09-11T07:57:13.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:13 smithi110 ceph-mon[25392]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.139", "placement": "1;smithi139=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:57:13.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:13 smithi110 ceph-mon[25392]: Saving service iscsi.datapool spec with placement smithi139=iscsi.a;count:1 2024-09-11T07:57:13.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:13 smithi110 ceph-mon[25392]: Deploying daemon iscsi.iscsi.a on smithi139 2024-09-11T07:57:13.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:13 smithi110 ceph-mon[25392]: pgmap v201: 132 pgs: 104 active+clean, 11 creating+peering, 17 unknown; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 12 KiB/s rd, 2.2 KiB/s wr, 24 op/s 2024-09-11T07:57:13.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:13 smithi110 ceph-mon[25392]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T07:57:13.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:13 smithi110 ceph-mon[25392]: Cluster is now healthy 2024-09-11T07:57:13.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:13 smithi110 ceph-mon[31709]: Checking dashboard <-> RGW credentials 2024-09-11T07:57:13.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:13 smithi110 ceph-mon[31709]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.139", "placement": "1;smithi139=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:57:13.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:13 smithi110 ceph-mon[31709]: Saving service iscsi.datapool spec with placement smithi139=iscsi.a;count:1 2024-09-11T07:57:13.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:13 smithi110 ceph-mon[31709]: Deploying daemon iscsi.iscsi.a on smithi139 2024-09-11T07:57:13.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:13 smithi110 ceph-mon[31709]: pgmap v201: 132 pgs: 104 active+clean, 11 creating+peering, 17 unknown; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 12 KiB/s rd, 2.2 KiB/s wr, 24 op/s 2024-09-11T07:57:13.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:13 smithi110 ceph-mon[31709]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T07:57:13.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:13 smithi110 ceph-mon[31709]: Cluster is now healthy 2024-09-11T07:57:13.974 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:57:13.980 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:13 smithi139 ceph-mon[27311]: Checking dashboard <-> RGW credentials 2024-09-11T07:57:13.980 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:13 smithi139 ceph-mon[27311]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.139", "placement": "1;smithi139=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:57:13.980 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:13 smithi139 ceph-mon[27311]: Saving service iscsi.datapool spec with placement smithi139=iscsi.a;count:1 2024-09-11T07:57:13.980 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:13 smithi139 ceph-mon[27311]: Deploying daemon iscsi.iscsi.a on smithi139 2024-09-11T07:57:13.980 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:13 smithi139 ceph-mon[27311]: pgmap v201: 132 pgs: 104 active+clean, 11 creating+peering, 17 unknown; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 12 KiB/s rd, 2.2 KiB/s wr, 24 op/s 2024-09-11T07:57:13.980 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:13 smithi139 ceph-mon[27311]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T07:57:13.980 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:13 smithi139 ceph-mon[27311]: Cluster is now healthy 2024-09-11T07:57:15.146 INFO:journalctl@ceph.iscsi.iscsi.a.smithi139.stdout:Sep 11 07:57:14 smithi139 podman[46641]: 2024-09-11 07:57:14.773468183 +0000 UTC m=+0.017141102 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T07:57:15.431 INFO:journalctl@ceph.iscsi.iscsi.a.smithi139.stdout:Sep 11 07:57:15 smithi139 podman[46641]: 2024-09-11 07:57:15.321453024 +0000 UTC m=+0.565125986 container create 0b960d67abc5b00df15d420264e6561b90867ed112c825ac471136bdc7e06fbf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True) 2024-09-11T07:57:15.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:15 smithi139 ceph-mon[27311]: pgmap v202: 132 pgs: 126 active+clean, 6 creating+peering; 581 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 62 KiB/s rd, 5.5 KiB/s wr, 145 op/s 2024-09-11T07:57:15.816 INFO:journalctl@ceph.iscsi.iscsi.a.smithi139.stdout:Sep 11 07:57:15 smithi139 podman[46641]: 2024-09-11 07:57:15.5317518 +0000 UTC m=+0.775424716 container init 0b960d67abc5b00df15d420264e6561b90867ed112c825ac471136bdc7e06fbf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-11T07:57:15.817 INFO:journalctl@ceph.iscsi.iscsi.a.smithi139.stdout:Sep 11 07:57:15 smithi139 podman[46641]: 2024-09-11 07:57:15.535330029 +0000 UTC m=+0.779002945 container start 0b960d67abc5b00df15d420264e6561b90867ed112c825ac471136bdc7e06fbf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T07:57:15.817 INFO:journalctl@ceph.iscsi.iscsi.a.smithi139.stdout:Sep 11 07:57:15 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu[46698]: log file path now is '/var/log/tcmu-runner.log' 2024-09-11T07:57:15.817 INFO:journalctl@ceph.iscsi.iscsi.a.smithi139.stdout:Sep 11 07:57:15 smithi139 bash[46641]: 0b960d67abc5b00df15d420264e6561b90867ed112c825ac471136bdc7e06fbf 2024-09-11T07:57:15.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:15 smithi110 ceph-mon[25392]: pgmap v202: 132 pgs: 126 active+clean, 6 creating+peering; 581 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 62 KiB/s rd, 5.5 KiB/s wr, 145 op/s 2024-09-11T07:57:15.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:15 smithi110 ceph-mon[31709]: pgmap v202: 132 pgs: 126 active+clean, 6 creating+peering; 581 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 62 KiB/s rd, 5.5 KiB/s wr, 145 op/s 2024-09-11T07:57:16.077 INFO:journalctl@ceph.iscsi.iscsi.a.smithi139.stdout:Sep 11 07:57:15 smithi139 podman[46740]: 2024-09-11 07:57:15.815996357 +0000 UTC m=+0.111733280 container create 0a249abecce95c0131e818145411b791f944a35874d09c69a76ea8186032f313 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-iscsi-iscsi-a, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-11T07:57:16.077 INFO:journalctl@ceph.iscsi.iscsi.a.smithi139.stdout:Sep 11 07:57:15 smithi139 podman[46740]: 2024-09-11 07:57:15.719912552 +0000 UTC m=+0.015649485 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T07:57:16.077 INFO:journalctl@ceph.iscsi.iscsi.a.smithi139.stdout:Sep 11 07:57:16 smithi139 podman[46740]: 2024-09-11 07:57:16.018337295 +0000 UTC m=+0.314074219 container init 0a249abecce95c0131e818145411b791f944a35874d09c69a76ea8186032f313 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-iscsi-iscsi-a, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-11T07:57:16.077 INFO:journalctl@ceph.iscsi.iscsi.a.smithi139.stdout:Sep 11 07:57:16 smithi139 podman[46740]: 2024-09-11 07:57:16.021456638 +0000 UTC m=+0.317193564 container start 0a249abecce95c0131e818145411b791f944a35874d09c69a76ea8186032f313 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-iscsi-iscsi-a, ceph=True, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-11T07:57:16.391 INFO:journalctl@ceph.iscsi.iscsi.a.smithi139.stdout:Sep 11 07:57:16 smithi139 bash[46740]: 0a249abecce95c0131e818145411b791f944a35874d09c69a76ea8186032f313 2024-09-11T07:57:16.392 INFO:journalctl@ceph.iscsi.iscsi.a.smithi139.stdout:Sep 11 07:57:16 smithi139 systemd[1]: Started Ceph iscsi.iscsi.a for 41bd1096-7012-11ef-bcea-c7b262605968. 2024-09-11T07:57:16.392 INFO:journalctl@ceph.iscsi.iscsi.a.smithi139.stdout:Sep 11 07:57:16 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-iscsi-iscsi-a[46755]: debug Started the configuration object watcher 2024-09-11T07:57:16.392 INFO:journalctl@ceph.iscsi.iscsi.a.smithi139.stdout:Sep 11 07:57:16 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-iscsi-iscsi-a[46755]: debug Checking for config object changes every 1s 2024-09-11T07:57:16.392 INFO:journalctl@ceph.iscsi.iscsi.a.smithi139.stdout:Sep 11 07:57:16 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-iscsi-iscsi-a[46755]: debug Processing osd blocklist entries for this node 2024-09-11T07:57:16.749 INFO:journalctl@ceph.iscsi.iscsi.a.smithi139.stdout:Sep 11 07:57:16 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-iscsi-iscsi-a[46755]: debug Reading the configuration object to update local LIO configuration 2024-09-11T07:57:16.749 INFO:journalctl@ceph.iscsi.iscsi.a.smithi139.stdout:Sep 11 07:57:16 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-iscsi-iscsi-a[46755]: debug Configuration does not have an entry for this host(smithi139.front.sepia.ceph.com) - nothing to define to LIO 2024-09-11T07:57:16.749 INFO:journalctl@ceph.iscsi.iscsi.a.smithi139.stdout:Sep 11 07:57:16 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-iscsi-iscsi-a[46755]: * Serving Flask app 'rbd-target-api' (lazy loading) 2024-09-11T07:57:16.749 INFO:journalctl@ceph.iscsi.iscsi.a.smithi139.stdout:Sep 11 07:57:16 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-iscsi-iscsi-a[46755]: * Environment: production 2024-09-11T07:57:16.749 INFO:journalctl@ceph.iscsi.iscsi.a.smithi139.stdout:Sep 11 07:57:16 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-iscsi-iscsi-a[46755]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-11T07:57:16.749 INFO:journalctl@ceph.iscsi.iscsi.a.smithi139.stdout:Sep 11 07:57:16 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-iscsi-iscsi-a[46755]: Use a production WSGI server instead. 2024-09-11T07:57:16.749 INFO:journalctl@ceph.iscsi.iscsi.a.smithi139.stdout:Sep 11 07:57:16 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-iscsi-iscsi-a[46755]: * Debug mode: off 2024-09-11T07:57:16.749 INFO:journalctl@ceph.iscsi.iscsi.a.smithi139.stdout:Sep 11 07:57:16 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-iscsi-iscsi-a[46755]: debug * Running on all addresses. 2024-09-11T07:57:16.749 INFO:journalctl@ceph.iscsi.iscsi.a.smithi139.stdout:Sep 11 07:57:16 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-iscsi-iscsi-a[46755]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-11T07:57:16.750 INFO:journalctl@ceph.iscsi.iscsi.a.smithi139.stdout:Sep 11 07:57:16 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-iscsi-iscsi-a[46755]: * Running on all addresses. 2024-09-11T07:57:16.750 INFO:journalctl@ceph.iscsi.iscsi.a.smithi139.stdout:Sep 11 07:57:16 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-iscsi-iscsi-a[46755]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-11T07:57:16.750 INFO:journalctl@ceph.iscsi.iscsi.a.smithi139.stdout:Sep 11 07:57:16 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-iscsi-iscsi-a[46755]: debug * Running on http://[fe80::ec4:7aff:fe88:7f59]:5000/ (Press CTRL+C to quit) 2024-09-11T07:57:16.750 INFO:journalctl@ceph.iscsi.iscsi.a.smithi139.stdout:Sep 11 07:57:16 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-iscsi-iscsi-a[46755]: * Running on http://[fe80::ec4:7aff:fe88:7f59]:5000/ (Press CTRL+C to quit) 2024-09-11T07:57:16.803 INFO:teuthology.orchestra.run.smithi139.stdout:Scheduled prometheus update... 2024-09-11T07:57:17.416 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:17 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:17.416 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:17 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:17.417 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:17 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:17.417 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:17 smithi139 ceph-mon[27311]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-11T07:57:17.417 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:17 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:17.417 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:17 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:57:17.417 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:17 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:17.417 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:17 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:57:17.417 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:17 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:17.417 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:17 smithi139 ceph-mon[27311]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-11T07:57:17.417 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:17 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2024-09-11T07:57:17.417 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:17 smithi139 ceph-mon[27311]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2024-09-11T07:57:17.417 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:17 smithi139 ceph-mon[27311]: Adding iSCSI gateway http://:@172.21.15.139:5000 to Dashboard 2024-09-11T07:57:17.417 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:17 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2024-09-11T07:57:17.418 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:17 smithi139 ceph-mon[27311]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2024-09-11T07:57:17.418 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:17 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:17.418 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:17 smithi139 ceph-mon[27311]: pgmap v203: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 68 KiB/s rd, 5.4 KiB/s wr, 161 op/s 2024-09-11T07:57:17.418 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:17 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi139"}]: dispatch 2024-09-11T07:57:17.418 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:17 smithi139 ceph-mon[27311]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi139"}]: dispatch 2024-09-11T07:57:17.418 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:17 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:17.418 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:17 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:57:17.418 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:17 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:17.418 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:17 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:57:17.418 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:17 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:17.418 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:17 smithi139 ceph-mon[27311]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-11T07:57:17.418 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:17 smithi139 ceph-mon[27311]: from='client.? 172.21.15.139:0/587079732' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-11T07:57:17.419 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:17 smithi139 ceph-mon[27311]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi139=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:57:17.419 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:17 smithi139 ceph-mon[27311]: Saving service prometheus spec with placement smithi139=a;count:1 2024-09-11T07:57:17.419 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:17 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:17.419 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:17 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:57:17.419 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:17 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:17.419 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:17 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:57:17.419 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:17 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:17.421 DEBUG:teuthology.orchestra.run.smithi139:prometheus.a> sudo journalctl -f -n 0 -u ceph-41bd1096-7012-11ef-bcea-c7b262605968@prometheus.a.service 2024-09-11T07:57:17.423 INFO:tasks.cephadm:Adding node-exporter.a on smithi110 2024-09-11T07:57:17.423 INFO:tasks.cephadm:Adding node-exporter.b on smithi139 2024-09-11T07:57:17.424 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- ceph orch apply node-exporter '2;smithi110=a;smithi139=b' 2024-09-11T07:57:17.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:17 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:17.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:17 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:17.469 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:17 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:17.469 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:17 smithi110 ceph-mon[25392]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-11T07:57:17.469 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:17 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:17.469 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:17 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:57:17.469 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:17 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:17.469 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:17 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:57:17.469 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:17 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:17.469 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:17 smithi110 ceph-mon[25392]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-11T07:57:17.469 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:17 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2024-09-11T07:57:17.469 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:17 smithi110 ceph-mon[25392]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2024-09-11T07:57:17.469 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:17 smithi110 ceph-mon[25392]: Adding iSCSI gateway http://:@172.21.15.139:5000 to Dashboard 2024-09-11T07:57:17.469 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:17 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2024-09-11T07:57:17.469 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:17 smithi110 ceph-mon[25392]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2024-09-11T07:57:17.470 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:17 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:17.470 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:17 smithi110 ceph-mon[25392]: pgmap v203: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 68 KiB/s rd, 5.4 KiB/s wr, 161 op/s 2024-09-11T07:57:17.470 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:17 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi139"}]: dispatch 2024-09-11T07:57:17.470 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:17 smithi110 ceph-mon[25392]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi139"}]: dispatch 2024-09-11T07:57:17.470 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:17 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:17.470 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:17 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:57:17.470 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:17 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:17.470 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:17 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:57:17.471 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:17 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:17.471 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:17 smithi110 ceph-mon[25392]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-11T07:57:17.471 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:17 smithi110 ceph-mon[25392]: from='client.? 172.21.15.139:0/587079732' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-11T07:57:17.471 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:17 smithi110 ceph-mon[25392]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi139=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:57:17.471 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:17 smithi110 ceph-mon[25392]: Saving service prometheus spec with placement smithi139=a;count:1 2024-09-11T07:57:17.471 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:17 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:17.471 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:17 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:17.471 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:17 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:17.471 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:17 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:17.472 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:17 smithi110 ceph-mon[31709]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-11T07:57:17.472 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:17 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:17.472 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:17 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:57:17.472 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:17 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:17.472 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:17 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:57:17.472 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:17 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:17.472 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:17 smithi110 ceph-mon[31709]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-11T07:57:17.472 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:17 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2024-09-11T07:57:17.472 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:17 smithi110 ceph-mon[31709]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2024-09-11T07:57:17.472 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:17 smithi110 ceph-mon[31709]: Adding iSCSI gateway http://:@172.21.15.139:5000 to Dashboard 2024-09-11T07:57:17.472 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:17 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2024-09-11T07:57:17.472 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:17 smithi110 ceph-mon[31709]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2024-09-11T07:57:17.472 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:17 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:17.473 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:17 smithi110 ceph-mon[31709]: pgmap v203: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 68 KiB/s rd, 5.4 KiB/s wr, 161 op/s 2024-09-11T07:57:17.473 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:17 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi139"}]: dispatch 2024-09-11T07:57:17.473 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:17 smithi110 ceph-mon[31709]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi139"}]: dispatch 2024-09-11T07:57:17.473 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:17 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:17.473 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:17 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:57:17.473 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:17 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:17.473 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:17 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:57:17.473 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:17 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:17.473 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:17 smithi110 ceph-mon[31709]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-11T07:57:17.473 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:17 smithi110 ceph-mon[31709]: from='client.? 172.21.15.139:0/587079732' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-11T07:57:17.473 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:17 smithi110 ceph-mon[31709]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi139=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:57:17.473 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:17 smithi110 ceph-mon[31709]: Saving service prometheus spec with placement smithi139=a;count:1 2024-09-11T07:57:17.474 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:17 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:17.474 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:17 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:57:17.474 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:17 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:17.474 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:17 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:57:17.474 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:17 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:17.475 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:17 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:57:17.475 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:17 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:17.475 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:17 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:57:17.475 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:17 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:18.440 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:18 smithi139 ceph-mon[27311]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-11T07:57:18.440 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:18 smithi139 ceph-mon[27311]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-11T07:57:18.440 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:18 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:18.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:18 smithi110 ceph-mon[25392]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-11T07:57:18.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:18 smithi110 ceph-mon[25392]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-11T07:57:18.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:18 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:18.468 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:18 smithi110 ceph-mon[31709]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-11T07:57:18.468 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:18 smithi110 ceph-mon[31709]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-11T07:57:18.468 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:18 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:18.482 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:57:19.448 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:19 smithi139 ceph-mon[27311]: pgmap v204: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 60 KiB/s rd, 4.6 KiB/s wr, 143 op/s 2024-09-11T07:57:19.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:19 smithi110 ceph-mon[25392]: pgmap v204: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 60 KiB/s rd, 4.6 KiB/s wr, 143 op/s 2024-09-11T07:57:19.468 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:19 smithi110 ceph-mon[31709]: pgmap v204: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 60 KiB/s rd, 4.6 KiB/s wr, 143 op/s 2024-09-11T07:57:20.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:20 smithi139 ceph-mon[27311]: osdmap e68: 8 total, 8 up, 8 in 2024-09-11T07:57:20.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:20 smithi110 ceph-mon[31709]: osdmap e68: 8 total, 8 up, 8 in 2024-09-11T07:57:20.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:20 smithi110 ceph-mon[25392]: osdmap e68: 8 total, 8 up, 8 in 2024-09-11T07:57:21.264 INFO:teuthology.orchestra.run.smithi139.stdout:Scheduled node-exporter update... 2024-09-11T07:57:21.626 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:21 smithi139 ceph-mon[27311]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 55 KiB/s rd, 4.1 KiB/s wr, 130 op/s 2024-09-11T07:57:21.626 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:21 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:21.626 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:21 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:21.626 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:21 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:57:21.626 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:21 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:21.626 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:21 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:57:21.627 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:21 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:21.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:21 smithi110 ceph-mon[25392]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 55 KiB/s rd, 4.1 KiB/s wr, 130 op/s 2024-09-11T07:57:21.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:21 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:21.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:21 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:21.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:21 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:57:21.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:21 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:21.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:21 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:57:21.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:21 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:21.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:21 smithi110 ceph-mon[31709]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 55 KiB/s rd, 4.1 KiB/s wr, 130 op/s 2024-09-11T07:57:21.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:21 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:21.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:21 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:21.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:21 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:57:21.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:21 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:21.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:21 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:57:21.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:21 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:22.145 DEBUG:teuthology.orchestra.run.smithi110:node-exporter.a> sudo journalctl -f -n 0 -u ceph-41bd1096-7012-11ef-bcea-c7b262605968@node-exporter.a.service 2024-09-11T07:57:22.148 DEBUG:teuthology.orchestra.run.smithi139:node-exporter.b> sudo journalctl -f -n 0 -u ceph-41bd1096-7012-11ef-bcea-c7b262605968@node-exporter.b.service 2024-09-11T07:57:22.150 INFO:tasks.cephadm:Adding alertmanager.a on smithi110 2024-09-11T07:57:22.150 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- ceph orch apply alertmanager '1;smithi110=a' 2024-09-11T07:57:22.468 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:22 smithi110 ceph-mon[31709]: from='client.24445 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi110=a;smithi139=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:57:22.468 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:22 smithi110 ceph-mon[31709]: Saving service node-exporter spec with placement smithi110=a;smithi139=b;count:2 2024-09-11T07:57:22.468 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:22 smithi110 ceph-mon[31709]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-11T07:57:22.468 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:22 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:22.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:22 smithi110 ceph-mon[25392]: from='client.24445 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi110=a;smithi139=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:57:22.469 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:22 smithi110 ceph-mon[25392]: Saving service node-exporter spec with placement smithi110=a;smithi139=b;count:2 2024-09-11T07:57:22.469 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:22 smithi110 ceph-mon[25392]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-11T07:57:22.469 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:22 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:22.524 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:57:22.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:22 smithi139 ceph-mon[27311]: from='client.24445 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi110=a;smithi139=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:57:22.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:22 smithi139 ceph-mon[27311]: Saving service node-exporter spec with placement smithi110=a;smithi139=b;count:2 2024-09-11T07:57:22.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:22 smithi139 ceph-mon[27311]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-11T07:57:22.645 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:22 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:23.637 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:23 smithi139 ceph-mon[27311]: pgmap v207: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 45 KiB/s rd, 3.3 KiB/s wr, 107 op/s 2024-09-11T07:57:23.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:23 smithi110 ceph-mon[25392]: pgmap v207: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 45 KiB/s rd, 3.3 KiB/s wr, 107 op/s 2024-09-11T07:57:23.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:23 smithi110 ceph-mon[31709]: pgmap v207: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 45 KiB/s rd, 3.3 KiB/s wr, 107 op/s 2024-09-11T07:57:24.311 INFO:teuthology.orchestra.run.smithi139.stdout:Scheduled alertmanager update... 2024-09-11T07:57:25.061 DEBUG:teuthology.orchestra.run.smithi110:alertmanager.a> sudo journalctl -f -n 0 -u ceph-41bd1096-7012-11ef-bcea-c7b262605968@alertmanager.a.service 2024-09-11T07:57:25.063 INFO:tasks.cephadm:Adding grafana.a on smithi139 2024-09-11T07:57:25.064 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- ceph orch apply grafana '1;smithi139=a' 2024-09-11T07:57:25.381 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:25 smithi110 ceph-mon[31709]: from='client.24451 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi110=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:57:25.381 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:25 smithi110 ceph-mon[31709]: Saving service alertmanager spec with placement smithi110=a;count:1 2024-09-11T07:57:25.381 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:25 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:25.381 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:25 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:57:25.382 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:25 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:25.382 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:25 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:57:25.382 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:25 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:25.382 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:25 smithi110 ceph-mon[31709]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-11T07:57:25.382 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:25 smithi110 ceph-mon[31709]: pgmap v208: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 14 KiB/s rd, 1023 B/s wr, 33 op/s 2024-09-11T07:57:25.382 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:25 smithi110 ceph-mon[25392]: from='client.24451 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi110=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:57:25.382 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:25 smithi110 ceph-mon[25392]: Saving service alertmanager spec with placement smithi110=a;count:1 2024-09-11T07:57:25.382 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:25 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:25.383 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:25 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:57:25.383 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:25 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:25.383 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:25 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:57:25.383 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:25 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:25.383 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:25 smithi110 ceph-mon[25392]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-11T07:57:25.383 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:25 smithi110 ceph-mon[25392]: pgmap v208: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 14 KiB/s rd, 1023 B/s wr, 33 op/s 2024-09-11T07:57:25.499 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:57:25.621 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:25 smithi139 ceph-mon[27311]: from='client.24451 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi110=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:57:25.621 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:25 smithi139 ceph-mon[27311]: Saving service alertmanager spec with placement smithi110=a;count:1 2024-09-11T07:57:25.622 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:25 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:25.622 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:25 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:57:25.622 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:25 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:25.622 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:25 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:57:25.622 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:25 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:25.622 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:25 smithi139 ceph-mon[27311]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-11T07:57:25.622 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:25 smithi139 ceph-mon[27311]: pgmap v208: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 14 KiB/s rd, 1023 B/s wr, 33 op/s 2024-09-11T07:57:26.587 INFO:journalctl@ceph.iscsi.iscsi.a.smithi139.stdout:Sep 11 07:57:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-iscsi-iscsi-a[46755]: debug there is no tcmu-runner data available 2024-09-11T07:57:26.588 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:26 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:26.588 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:26 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:26.588 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:26 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:57:26.588 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:26 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:26.588 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:26 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:26.588 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:26 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:57:26.588 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:26 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:26.588 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:26 smithi139 ceph-mon[27311]: Deploying daemon prometheus.a on smithi139 2024-09-11T07:57:26.684 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:26 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:26.684 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:26 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:26.684 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:26 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:57:26.684 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:26 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:26.684 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:26 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:26.684 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:26 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:57:26.684 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:26 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:26.684 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:26 smithi110 ceph-mon[31709]: Deploying daemon prometheus.a on smithi139 2024-09-11T07:57:26.685 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:26 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:26.685 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:26 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:26.685 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:26 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:57:26.685 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:26 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:26.685 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:26 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:26.685 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:26 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:57:26.685 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:26 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:26.685 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:26 smithi110 ceph-mon[25392]: Deploying daemon prometheus.a on smithi139 2024-09-11T07:57:27.225 INFO:teuthology.orchestra.run.smithi139.stdout:Scheduled grafana update... 2024-09-11T07:57:27.598 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:27 smithi110 ceph-mon[25392]: from='client.24449 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:57:27.598 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:27 smithi110 ceph-mon[25392]: pgmap v209: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.6 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-11T07:57:27.598 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:27 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:27.598 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:27 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:27.598 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:27 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:27.598 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:27 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:27.598 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:27 smithi110 ceph-mon[31709]: from='client.24449 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:57:27.598 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:27 smithi110 ceph-mon[31709]: pgmap v209: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.6 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-11T07:57:27.599 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:27 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:27.599 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:27 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:27.599 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:27 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:27.599 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:27 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:27.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:27 smithi139 ceph-mon[27311]: from='client.24449 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:57:27.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:27 smithi139 ceph-mon[27311]: pgmap v209: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.6 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-11T07:57:27.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:27 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:27.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:27 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:27.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:27 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:27.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:27 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:27.886 DEBUG:teuthology.orchestra.run.smithi139:grafana.a> sudo journalctl -f -n 0 -u ceph-41bd1096-7012-11ef-bcea-c7b262605968@grafana.a.service 2024-09-11T07:57:27.888 INFO:tasks.cephadm:Setting up client nodes... 2024-09-11T07:57:27.888 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-11T07:57:28.216 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:57:28.688 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:28 smithi110 ceph-mon[25392]: from='client.24457 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi139=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:57:28.688 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:28 smithi110 ceph-mon[25392]: Saving service grafana spec with placement smithi139=a;count:1 2024-09-11T07:57:28.689 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:28 smithi110 ceph-mon[31709]: from='client.24457 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi139=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:57:28.689 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:28 smithi110 ceph-mon[31709]: Saving service grafana spec with placement smithi139=a;count:1 2024-09-11T07:57:28.998 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:28 smithi139 ceph-mon[27311]: from='client.24457 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi139=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:57:28.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:28 smithi139 ceph-mon[27311]: Saving service grafana spec with placement smithi139=a;count:1 2024-09-11T07:57:29.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:29 smithi110 ceph-mon[25392]: pgmap v210: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 818 B/s rd, 102 B/s wr, 1 op/s 2024-09-11T07:57:29.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:29 smithi110 ceph-mon[31709]: pgmap v210: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 818 B/s rd, 102 B/s wr, 1 op/s 2024-09-11T07:57:29.998 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:29 smithi139 ceph-mon[27311]: pgmap v210: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 818 B/s rd, 102 B/s wr, 1 op/s 2024-09-11T07:57:30.208 INFO:teuthology.orchestra.run.smithi110.stdout:[client.0] 2024-09-11T07:57:30.209 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQBqTeFm8Yc0DBAAksvOq0x7EG2XQTAaop+l9g== 2024-09-11T07:57:30.856 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:30 smithi110 ceph-mon[31709]: from='client.? 172.21.15.110:0/3229665486' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T07:57:30.856 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:30 smithi110 ceph-mon[31709]: from='client.? 172.21.15.110:0/3229665486' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T07:57:30.857 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:30 smithi110 ceph-mon[25392]: from='client.? 172.21.15.110:0/3229665486' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T07:57:30.857 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:30 smithi110 ceph-mon[25392]: from='client.? 172.21.15.110:0/3229665486' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T07:57:30.879 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-11T07:57:30.879 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-11T07:57:30.879 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-11T07:57:30.918 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-11T07:57:30.944 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:30 smithi139 ceph-mon[27311]: from='client.? 172.21.15.110:0/3229665486' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T07:57:30.945 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:30 smithi139 ceph-mon[27311]: from='client.? 172.21.15.110:0/3229665486' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T07:57:31.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:31 smithi110 ceph-mon[31709]: pgmap v211: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 930 B/s rd, 0 op/s 2024-09-11T07:57:31.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:31 smithi110 ceph-mon[25392]: pgmap v211: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 930 B/s rd, 0 op/s 2024-09-11T07:57:31.998 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:31 smithi139 ceph-mon[27311]: pgmap v211: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 930 B/s rd, 0 op/s 2024-09-11T07:57:32.325 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:57:32.842 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:32 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [7, 2]}]: dispatch 2024-09-11T07:57:32.843 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:32 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2024-09-11T07:57:32.843 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:32 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2024-09-11T07:57:32.843 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:32 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2024-09-11T07:57:32.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:32 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [7, 2]}]: dispatch 2024-09-11T07:57:32.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:32 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2024-09-11T07:57:32.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:32 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2024-09-11T07:57:32.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:32 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2024-09-11T07:57:32.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:32 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [7, 2]}]: dispatch 2024-09-11T07:57:32.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:32 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2024-09-11T07:57:32.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:32 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2024-09-11T07:57:32.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:32 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2024-09-11T07:57:33.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:33 smithi110 ceph-mon[25392]: pgmap v212: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:57:33.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:33 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [7, 2]}]': finished 2024-09-11T07:57:33.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:33 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]': finished 2024-09-11T07:57:33.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:33 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]': finished 2024-09-11T07:57:33.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:33 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]': finished 2024-09-11T07:57:33.969 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:33 smithi110 ceph-mon[25392]: osdmap e69: 8 total, 8 up, 8 in 2024-09-11T07:57:33.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:33 smithi110 ceph-mon[31709]: pgmap v212: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:57:33.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:33 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [7, 2]}]': finished 2024-09-11T07:57:33.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:33 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]': finished 2024-09-11T07:57:33.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:33 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]': finished 2024-09-11T07:57:33.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:33 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]': finished 2024-09-11T07:57:33.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:33 smithi110 ceph-mon[31709]: osdmap e69: 8 total, 8 up, 8 in 2024-09-11T07:57:33.998 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:33 smithi139 ceph-mon[27311]: pgmap v212: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:57:33.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:33 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [7, 2]}]': finished 2024-09-11T07:57:33.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:33 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]': finished 2024-09-11T07:57:33.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:33 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]': finished 2024-09-11T07:57:33.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:33 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]': finished 2024-09-11T07:57:33.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:33 smithi139 ceph-mon[27311]: osdmap e69: 8 total, 8 up, 8 in 2024-09-11T07:57:34.748 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:34 smithi139 ceph-mon[27311]: osdmap e70: 8 total, 8 up, 8 in 2024-09-11T07:57:34.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:34 smithi110 ceph-mon[25392]: osdmap e70: 8 total, 8 up, 8 in 2024-09-11T07:57:34.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:34 smithi110 ceph-mon[31709]: osdmap e70: 8 total, 8 up, 8 in 2024-09-11T07:57:35.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:35 smithi139 ceph-mon[27311]: pgmap v215: 132 pgs: 1 peering, 131 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:57:35.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:35 smithi110 ceph-mon[25392]: pgmap v215: 132 pgs: 1 peering, 131 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:57:35.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:35 smithi110 ceph-mon[31709]: pgmap v215: 132 pgs: 1 peering, 131 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:57:36.638 INFO:journalctl@ceph.iscsi.iscsi.a.smithi139.stdout:Sep 11 07:57:36 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-iscsi-iscsi-a[46755]: debug there is no tcmu-runner data available 2024-09-11T07:57:36.957 INFO:teuthology.orchestra.run.smithi139.stdout:[client.1] 2024-09-11T07:57:36.958 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQBwTeFmu5LROBAAGUfeRi7WJs9viz/Gv9UOYg== 2024-09-11T07:57:37.681 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:37 smithi139 ceph-mon[27311]: pgmap v216: 132 pgs: 1 peering, 131 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 27 B/s, 1 objects/s recovering 2024-09-11T07:57:37.682 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:37 smithi139 ceph-mon[27311]: from='client.24449 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:57:37.682 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:37 smithi139 ceph-mon[27311]: from='client.? 172.21.15.139:0/1379095842' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T07:57:37.682 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:37 smithi139 ceph-mon[27311]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T07:57:37.682 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:37 smithi139 ceph-mon[27311]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T07:57:37.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:37 smithi110 ceph-mon[25392]: pgmap v216: 132 pgs: 1 peering, 131 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 27 B/s, 1 objects/s recovering 2024-09-11T07:57:37.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:37 smithi110 ceph-mon[25392]: from='client.24449 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:57:37.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:37 smithi110 ceph-mon[25392]: from='client.? 172.21.15.139:0/1379095842' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T07:57:37.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:37 smithi110 ceph-mon[25392]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T07:57:37.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:37 smithi110 ceph-mon[25392]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T07:57:37.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:37 smithi110 ceph-mon[31709]: pgmap v216: 132 pgs: 1 peering, 131 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 27 B/s, 1 objects/s recovering 2024-09-11T07:57:37.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:37 smithi110 ceph-mon[31709]: from='client.24449 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:57:37.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:37 smithi110 ceph-mon[31709]: from='client.? 172.21.15.139:0/1379095842' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T07:57:37.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:37 smithi110 ceph-mon[31709]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T07:57:37.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:37 smithi110 ceph-mon[31709]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T07:57:38.249 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:57:37 smithi139 systemd[1]: Starting Ceph prometheus.a for 41bd1096-7012-11ef-bcea-c7b262605968... 2024-09-11T07:57:39.748 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:39 smithi139 ceph-mon[27311]: pgmap v217: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 72 B/s, 2 objects/s recovering 2024-09-11T07:57:39.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:39 smithi110 ceph-mon[25392]: pgmap v217: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 72 B/s, 2 objects/s recovering 2024-09-11T07:57:39.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:39 smithi110 ceph-mon[31709]: pgmap v217: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 72 B/s, 2 objects/s recovering 2024-09-11T07:57:39.984 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-11T07:57:39.984 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-09-11T07:57:39.984 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-11T07:57:40.062 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-11T07:57:40.062 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-11T07:57:40.063 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- ceph mgr dump --format=json 2024-09-11T07:57:40.325 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:57:41.260 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:57:41 smithi139 podman[48858]: 2024-09-11 07:57:41.007151483 +0000 UTC m=+0.103834973 container create 5065c2f188985fc64af6e9c6c30a755b18073762ab57a25dbef95a1f019b5c39 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T07:57:41.260 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:57:41 smithi139 podman[48858]: 2024-09-11 07:57:40.921103785 +0000 UTC m=+0.017787239 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-09-11T07:57:41.260 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:57:41 smithi139 podman[48858]: 2024-09-11 07:57:41.258970182 +0000 UTC m=+0.355653627 container init 5065c2f188985fc64af6e9c6c30a755b18073762ab57a25dbef95a1f019b5c39 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T07:57:41.591 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:57:41 smithi139 podman[48858]: 2024-09-11 07:57:41.26222358 +0000 UTC m=+0.358907024 container start 5065c2f188985fc64af6e9c6c30a755b18073762ab57a25dbef95a1f019b5c39 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T07:57:41.591 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:57:41 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a[48872]: ts=2024-09-11T07:57:41.287Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-09-11T07:57:41.591 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:57:41 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a[48872]: ts=2024-09-11T07:57:41.287Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-09-11T07:57:41.592 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:57:41 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a[48872]: ts=2024-09-11T07:57:41.287Z caller=main.go:570 level=info host_details="(Linux 5.14.0-505.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Sep 5 07:54:07 UTC 2024 x86_64 smithi139 (none))" 2024-09-11T07:57:41.592 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:57:41 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a[48872]: ts=2024-09-11T07:57:41.287Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-09-11T07:57:41.592 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:57:41 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a[48872]: ts=2024-09-11T07:57:41.287Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-09-11T07:57:41.592 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:57:41 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a[48872]: ts=2024-09-11T07:57:41.288Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-09-11T07:57:41.592 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:57:41 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a[48872]: ts=2024-09-11T07:57:41.289Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-09-11T07:57:41.592 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:57:41 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a[48872]: ts=2024-09-11T07:57:41.289Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-09-11T07:57:41.592 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:57:41 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a[48872]: ts=2024-09-11T07:57:41.290Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-09-11T07:57:41.592 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:57:41 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a[48872]: ts=2024-09-11T07:57:41.292Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-09-11T07:57:41.592 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:57:41 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a[48872]: ts=2024-09-11T07:57:41.292Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.774µs 2024-09-11T07:57:41.592 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:57:41 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a[48872]: ts=2024-09-11T07:57:41.292Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-09-11T07:57:41.592 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:57:41 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a[48872]: ts=2024-09-11T07:57:41.293Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0 2024-09-11T07:57:41.592 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:57:41 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a[48872]: ts=2024-09-11T07:57:41.293Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=17.562µs wal_replay_duration=379.038µs wbl_replay_duration=116ns total_replay_duration=413.794µs 2024-09-11T07:57:41.593 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:57:41 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a[48872]: ts=2024-09-11T07:57:41.293Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-09-11T07:57:41.593 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:57:41 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a[48872]: ts=2024-09-11T07:57:41.293Z caller=main.go:1029 level=info msg="TSDB started" 2024-09-11T07:57:41.593 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:57:41 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a[48872]: ts=2024-09-11T07:57:41.293Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-09-11T07:57:41.593 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:57:41 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a[48872]: ts=2024-09-11T07:57:41.307Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=13.512705ms db_storage=697ns remote_storage=961ns web_handler=350ns query_engine=568ns scrape=444.926µs scrape_sd=17.263µs notify=1.035µs notify_sd=1.212µs rules=12.796696ms tracing=5.543µs 2024-09-11T07:57:41.593 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:57:41 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a[48872]: ts=2024-09-11T07:57:41.307Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-09-11T07:57:41.593 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:57:41 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a[48872]: ts=2024-09-11T07:57:41.307Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-09-11T07:57:41.593 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:57:41 smithi139 bash[48858]: 5065c2f188985fc64af6e9c6c30a755b18073762ab57a25dbef95a1f019b5c39 2024-09-11T07:57:41.593 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:57:41 smithi139 systemd[1]: Started Ceph prometheus.a for 41bd1096-7012-11ef-bcea-c7b262605968. 2024-09-11T07:57:41.739 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:41 smithi110 ceph-mon[25392]: pgmap v218: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 72 B/s, 2 objects/s recovering 2024-09-11T07:57:41.740 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:41 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:41.740 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:41 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:41.740 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:41 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:41.740 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:41 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-11T07:57:41.740 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:41 smithi110 ceph-mon[31709]: pgmap v218: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 72 B/s, 2 objects/s recovering 2024-09-11T07:57:41.740 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:41 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:41.740 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:41 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:41.740 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:41 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:41.740 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:41 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-11T07:57:41.872 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:41 smithi139 ceph-mon[27311]: pgmap v218: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 72 B/s, 2 objects/s recovering 2024-09-11T07:57:41.872 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:41 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:41.872 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:41 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:41.873 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:41 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:41.873 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:41 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-11T07:57:42.231 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T07:57:42.746 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:57:42 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x[29520]: ignoring --setuser ceph since I am not root 2024-09-11T07:57:42.746 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:57:42 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x[29520]: ignoring --setgroup ceph since I am not root 2024-09-11T07:57:42.746 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:57:42 smithi139 ceph-mgr[29524]: -- 172.21.15.139:0/2755163086 <== mon.2 v2:172.21.15.110:3301/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5580df8e2340 con 0x5580df8ef000 2024-09-11T07:57:42.746 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:57:42 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x[29520]: 2024-09-11T07:57:42.616+0000 7f3684342ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T07:57:42.746 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:57:42 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x[29520]: 2024-09-11T07:57:42.677+0000 7f3684342ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T07:57:42.747 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:57:42 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x[29520]: 2024-09-11T07:57:42.744+0000 7f3684342ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T07:57:42.840 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:57:42 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: ignoring --setuser ceph since I am not root 2024-09-11T07:57:42.840 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:57:42 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: ignoring --setgroup ceph since I am not root 2024-09-11T07:57:42.840 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:57:42 smithi110 ceph-mgr[25622]: -- 172.21.15.110:0/4037381491 <== mon.0 v2:172.21.15.110:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ceddec8340 con 0x55cedded4c00 2024-09-11T07:57:42.840 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:57:42 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:57:42.595+0000 7f8659119ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T07:57:42.840 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:57:42 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:57:42.651+0000 7f8659119ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T07:57:42.840 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:57:42 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:57:42.716+0000 7f8659119ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T07:57:42.841 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:57:42 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:57:42.838+0000 7f8659119ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T07:57:42.998 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:57:42 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x[29520]: 2024-09-11T07:57:42.871+0000 7f3684342ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T07:57:43.102 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:57:42 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:57:42.973+0000 7f8659119ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T07:57:43.103 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":16,"flags":0,"active_gid":14150,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":248793870},{"type":"v1","addr":"172.21.15.110:6801","nonce":248793870}]},"active_addr":"172.21.15.110:6801/248793870","active_change":"2024-09-11T07:52:32.111559+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":24112,"name":"x","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.110:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"cephadm","addrvec":[{"type":"v2","addr":"172.21.15.110:0","nonce":2224974349}]},{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.110:0","nonce":1006666897}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.110:0","nonce":1550053204}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.110:0","nonce":4293537776}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.110:0","nonce":3473535153}]}]} 2024-09-11T07:57:43.107 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-11T07:57:43.108 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-11T07:57:43.108 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T07:57:43.277 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:57:43 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x[29520]: 2024-09-11T07:57:43.006+0000 7f3684342ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T07:57:43.277 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:43 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:43.278 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:43 smithi139 ceph-mon[27311]: from='client.? 172.21.15.110:0/304364152' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-11T07:57:43.278 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:43 smithi139 ceph-mon[27311]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-11T07:57:43.278 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:43 smithi139 ceph-mon[27311]: mgrmap e17: y(active, since 5m), standbys: x 2024-09-11T07:57:43.357 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:57:43.468 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:57:43 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:57:43.238+0000 7f8659119ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T07:57:43.468 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:57:43 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:57:43.302+0000 7f8659119ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T07:57:43.468 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:43 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:43.468 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:43 smithi110 ceph-mon[31709]: from='client.? 172.21.15.110:0/304364152' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-11T07:57:43.468 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:43 smithi110 ceph-mon[31709]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-11T07:57:43.468 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:43 smithi110 ceph-mon[31709]: mgrmap e17: y(active, since 5m), standbys: x 2024-09-11T07:57:43.469 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:43 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' 2024-09-11T07:57:43.469 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:43 smithi110 ceph-mon[25392]: from='client.? 172.21.15.110:0/304364152' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-11T07:57:43.469 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:43 smithi110 ceph-mon[25392]: from='mgr.14150 172.21.15.110:0/3145787425' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-11T07:57:43.469 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:43 smithi110 ceph-mon[25392]: mgrmap e17: y(active, since 5m), standbys: x 2024-09-11T07:57:43.539 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:57:43 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x[29520]: 2024-09-11T07:57:43.276+0000 7f3684342ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T07:57:43.539 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:57:43 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x[29520]: 2024-09-11T07:57:43.344+0000 7f3684342ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T07:57:43.805 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:57:43 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x[29520]: 2024-09-11T07:57:43.537+0000 7f3684342ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T07:57:43.808 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:57:43 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:57:43.474+0000 7f8659119ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T07:57:44.718 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:57:44 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:57:44.333+0000 7f8659119ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T07:57:44.718 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:57:44 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:57:44.398+0000 7f8659119ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T07:57:44.740 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:57:44 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x[29520]: 2024-09-11T07:57:44.462+0000 7f3684342ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T07:57:44.741 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:57:44 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x[29520]: 2024-09-11T07:57:44.529+0000 7f3684342ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T07:57:45.285 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:57:45 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:57:45.126+0000 7f8659119ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T07:57:45.286 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:57:45 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:57:45.242+0000 7f8659119ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T07:57:45.288 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T07:57:45.289 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":70,"fsid":"41bd1096-7012-11ef-bcea-c7b262605968","created":"2024-09-11T07:51:48.770375+0000","modified":"2024-09-11T07:57:33.569220+0000","last_up_change":"2024-09-11T07:56:50.396037+0000","last_in_change":"2024-09-11T07:56:31.845597+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T07:54:50.418595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-09-11T07:57:02.758662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":3,"pool_name":"datapool","create_time":"2024-09-11T07:57:02.991039+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":65,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-11T07:57:04.775543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-11T07:57:06.736275+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-11T07:57:08.802758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}}}],"osds":[{"osd":0,"uuid":"05f2cdd1-77dc-4b26-afa8-16099c528c10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":2716989082},{"type":"v1","addr":"172.21.15.110:6803","nonce":2716989082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":2716989082},{"type":"v1","addr":"172.21.15.110:6805","nonce":2716989082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":2716989082},{"type":"v1","addr":"172.21.15.110:6809","nonce":2716989082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":2716989082},{"type":"v1","addr":"172.21.15.110:6807","nonce":2716989082}]},"public_addr":"172.21.15.110:6803/2716989082","cluster_addr":"172.21.15.110:6805/2716989082","heartbeat_back_addr":"172.21.15.110:6809/2716989082","heartbeat_front_addr":"172.21.15.110:6807/2716989082","state":["exists","up"]},{"osd":1,"uuid":"1e01c286-b92c-46b3-b61c-f8aea652b8be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":1928520079},{"type":"v1","addr":"172.21.15.110:6811","nonce":1928520079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":1928520079},{"type":"v1","addr":"172.21.15.110:6813","nonce":1928520079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":1928520079},{"type":"v1","addr":"172.21.15.110:6817","nonce":1928520079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":1928520079},{"type":"v1","addr":"172.21.15.110:6815","nonce":1928520079}]},"public_addr":"172.21.15.110:6811/1928520079","cluster_addr":"172.21.15.110:6813/1928520079","heartbeat_back_addr":"172.21.15.110:6817/1928520079","heartbeat_front_addr":"172.21.15.110:6815/1928520079","state":["exists","up"]},{"osd":2,"uuid":"b413fbb5-f09d-4a04-a72d-1cc5d554f90d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":3894315382},{"type":"v1","addr":"172.21.15.110:6819","nonce":3894315382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":3894315382},{"type":"v1","addr":"172.21.15.110:6821","nonce":3894315382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":3894315382},{"type":"v1","addr":"172.21.15.110:6825","nonce":3894315382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":3894315382},{"type":"v1","addr":"172.21.15.110:6823","nonce":3894315382}]},"public_addr":"172.21.15.110:6819/3894315382","cluster_addr":"172.21.15.110:6821/3894315382","heartbeat_back_addr":"172.21.15.110:6825/3894315382","heartbeat_front_addr":"172.21.15.110:6823/3894315382","state":["exists","up"]},{"osd":3,"uuid":"293d5b4d-11ef-4d6a-9b65-5a6b47472c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":1640229305},{"type":"v1","addr":"172.21.15.110:6827","nonce":1640229305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":1640229305},{"type":"v1","addr":"172.21.15.110:6829","nonce":1640229305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":1640229305},{"type":"v1","addr":"172.21.15.110:6833","nonce":1640229305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":1640229305},{"type":"v1","addr":"172.21.15.110:6831","nonce":1640229305}]},"public_addr":"172.21.15.110:6827/1640229305","cluster_addr":"172.21.15.110:6829/1640229305","heartbeat_back_addr":"172.21.15.110:6833/1640229305","heartbeat_front_addr":"172.21.15.110:6831/1640229305","state":["exists","up"]},{"osd":4,"uuid":"f9119e96-9863-45b7-b2d5-89bc3eeb1fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2399332353},{"type":"v1","addr":"172.21.15.139:6801","nonce":2399332353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2399332353},{"type":"v1","addr":"172.21.15.139:6803","nonce":2399332353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2399332353},{"type":"v1","addr":"172.21.15.139:6807","nonce":2399332353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2399332353},{"type":"v1","addr":"172.21.15.139:6805","nonce":2399332353}]},"public_addr":"172.21.15.139:6801/2399332353","cluster_addr":"172.21.15.139:6803/2399332353","heartbeat_back_addr":"172.21.15.139:6807/2399332353","heartbeat_front_addr":"172.21.15.139:6805/2399332353","state":["exists","up"]},{"osd":5,"uuid":"3339eb92-7017-4bb8-947f-557a32723e94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1641418305},{"type":"v1","addr":"172.21.15.139:6809","nonce":1641418305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1641418305},{"type":"v1","addr":"172.21.15.139:6811","nonce":1641418305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1641418305},{"type":"v1","addr":"172.21.15.139:6815","nonce":1641418305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1641418305},{"type":"v1","addr":"172.21.15.139:6813","nonce":1641418305}]},"public_addr":"172.21.15.139:6809/1641418305","cluster_addr":"172.21.15.139:6811/1641418305","heartbeat_back_addr":"172.21.15.139:6815/1641418305","heartbeat_front_addr":"172.21.15.139:6813/1641418305","state":["exists","up"]},{"osd":6,"uuid":"69a4457c-39cc-41f5-bceb-9229b8b030f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1783109268},{"type":"v1","addr":"172.21.15.139:6817","nonce":1783109268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1783109268},{"type":"v1","addr":"172.21.15.139:6819","nonce":1783109268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1783109268},{"type":"v1","addr":"172.21.15.139:6823","nonce":1783109268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1783109268},{"type":"v1","addr":"172.21.15.139:6821","nonce":1783109268}]},"public_addr":"172.21.15.139:6817/1783109268","cluster_addr":"172.21.15.139:6819/1783109268","heartbeat_back_addr":"172.21.15.139:6823/1783109268","heartbeat_front_addr":"172.21.15.139:6821/1783109268","state":["exists","up"]},{"osd":7,"uuid":"c58219c0-3caa-4b99-afb1-bc572ac152ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":232507052},{"type":"v1","addr":"172.21.15.139:6825","nonce":232507052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":232507052},{"type":"v1","addr":"172.21.15.139:6827","nonce":232507052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":232507052},{"type":"v1","addr":"172.21.15.139:6831","nonce":232507052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":232507052},{"type":"v1","addr":"172.21.15.139:6829","nonce":232507052}]},"public_addr":"172.21.15.139:6825/232507052","cluster_addr":"172.21.15.139:6827/232507052","heartbeat_back_addr":"172.21.15.139:6831/232507052","heartbeat_front_addr":"172.21.15.139:6829/232507052","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:53:57.309408+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:54:22.407580+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:54:47.670630+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:55:13.231294+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:55:37.035454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:56:00.079531+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:56:24.721942+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:56:48.981334+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":7,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.110:6801/4096534802":"2024-09-12T07:52:32.111455+0000","172.21.15.110:0/2391676173":"2024-09-12T07:52:32.111455+0000","172.21.15.110:0/1603882420":"2024-09-12T07:52:32.111455+0000","172.21.15.110:6800/4096534802":"2024-09-12T07:52:32.111455+0000","172.21.15.110:0/2727007179":"2024-09-12T07:52:10.347528+0000","172.21.15.110:0/1589810572":"2024-09-12T07:52:10.347528+0000","172.21.15.110:0/828045681":"2024-09-12T07:52:10.347528+0000","172.21.15.110:6801/1242491477":"2024-09-12T07:52:10.347528+0000","172.21.15.110:0/3707065533":"2024-09-12T07:52:32.111455+0000","172.21.15.110:6800/1242491477":"2024-09-12T07:52:10.347528+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T07:57:45.528 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:45 smithi139 ceph-mon[27311]: from='client.? 172.21.15.110:0/4148936178' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T07:57:45.528 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:57:45 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x[29520]: 2024-09-11T07:57:45.260+0000 7f3684342ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T07:57:45.528 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:57:45 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x[29520]: 2024-09-11T07:57:45.380+0000 7f3684342ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T07:57:45.528 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:57:45 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x[29520]: 2024-09-11T07:57:45.504+0000 7f3684342ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T07:57:45.562 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:45 smithi110 ceph-mon[25392]: from='client.? 172.21.15.110:0/4148936178' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T07:57:45.563 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:45 smithi110 ceph-mon[31709]: from='client.? 172.21.15.110:0/4148936178' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T07:57:45.563 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:57:45 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:57:45.369+0000 7f8659119ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T07:57:45.563 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:57:45 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:57:45.433+0000 7f8659119ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T07:57:45.563 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:57:45 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:57:45.561+0000 7f8659119ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T07:57:45.829 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:57:45 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x[29520]: 2024-09-11T07:57:45.566+0000 7f3684342ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T07:57:45.829 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:57:45 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x[29520]: 2024-09-11T07:57:45.697+0000 7f3684342ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T07:57:45.829 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:57:45 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x[29520]: 2024-09-11T07:57:45.766+0000 7f3684342ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T07:57:45.829 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:57:45 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x[29520]: 2024-09-11T07:57:45.827+0000 7f3684342ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T07:57:45.829 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:57:45 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:57:45.630+0000 7f8659119ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T07:57:45.832 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:57:45 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:57:45.689+0000 7f8659119ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T07:57:46.090 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:57:45 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x[29520]: 2024-09-11T07:57:45.961+0000 7f3684342ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T07:57:46.090 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:57:46 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x[29520]: 2024-09-11T07:57:46.024+0000 7f3684342ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T07:57:46.103 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:57:45 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:57:45.828+0000 7f8659119ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T07:57:46.103 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:57:45 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:57:45.893+0000 7f8659119ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T07:57:46.103 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:57:45 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:57:45.958+0000 7f8659119ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T07:57:46.103 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:57:46 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:57:46.041+0000 7f8659119ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T07:57:46.159 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-11T07:57:46.159 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T07:57:46.368 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:57:46 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:57:46.102+0000 7f8659119ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T07:57:46.368 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:57:46 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:57:46.162+0000 7f8659119ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T07:57:46.498 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:57:46 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x[29520]: 2024-09-11T07:57:46.088+0000 7f3684342ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T07:57:46.498 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:57:46 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x[29520]: 2024-09-11T07:57:46.168+0000 7f3684342ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T07:57:46.499 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:57:46 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x[29520]: 2024-09-11T07:57:46.224+0000 7f3684342ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T07:57:46.499 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:57:46 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x[29520]: 2024-09-11T07:57:46.280+0000 7f3684342ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T07:57:46.968 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:57:46 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:57:46.933+0000 7f8659119ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T07:57:47.249 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:57:47 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x[29520]: 2024-09-11T07:57:47.028+0000 7f3684342ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T07:57:47.249 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:57:47 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x[29520]: [11/Sep/2024:07:57:47] ENGINE Bus STARTING 2024-09-11T07:57:47.249 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:57:47 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x[29520]: CherryPy Checker: 2024-09-11T07:57:47.249 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:57:47 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x[29520]: The Application mounted at '' has an empty config. 2024-09-11T07:57:47.249 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:57:47 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x[29520]: 2024-09-11T07:57:47.249 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:57:47 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x[29520]: [11/Sep/2024:07:57:47] ENGINE Serving on http://:::9283 2024-09-11T07:57:47.249 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:57:47 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x[29520]: [11/Sep/2024:07:57:47] ENGINE Bus STARTED 2024-09-11T07:57:47.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:46 smithi139 ceph-mon[27311]: Active manager daemon y restarted 2024-09-11T07:57:47.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:46 smithi139 ceph-mon[27311]: Activating manager daemon y 2024-09-11T07:57:47.250 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:46 smithi139 ceph-mon[27311]: osdmap e71: 8 total, 8 up, 8 in 2024-09-11T07:57:47.250 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:46 smithi139 ceph-mon[27311]: mgrmap e18: y(active, starting, since 0.00975949s), standbys: x 2024-09-11T07:57:47.250 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:46 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:57:47.250 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:46 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:57:47.250 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:46 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:57:47.250 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:46 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T07:57:47.250 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:46 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T07:57:47.250 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:46 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:57:47.250 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:46 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:57:47.250 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:46 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:57:47.250 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:46 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:57:47.250 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:46 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:57:47.250 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:46 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:57:47.250 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:46 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:57:47.251 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:46 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:57:47.251 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:46 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T07:57:47.251 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:46 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T07:57:47.251 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:46 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T07:57:47.251 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:46 smithi139 ceph-mon[27311]: Manager daemon y is now available 2024-09-11T07:57:47.251 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:46 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:57:47.251 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:46 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:57:47.255 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:57:47 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: [11/Sep/2024:07:57:47] ENGINE Bus STARTING 2024-09-11T07:57:47.255 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:57:47 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: CherryPy Checker: 2024-09-11T07:57:47.255 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:57:47 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: The Application mounted at '' has an empty config. 2024-09-11T07:57:47.255 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:57:47 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:57:47.256 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:57:47 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: [11/Sep/2024:07:57:47] ENGINE Serving on http://:::9283 2024-09-11T07:57:47.256 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:57:47 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: [11/Sep/2024:07:57:47] ENGINE Bus STARTED 2024-09-11T07:57:47.256 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:46 smithi110 ceph-mon[31709]: Active manager daemon y restarted 2024-09-11T07:57:47.256 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:46 smithi110 ceph-mon[31709]: Activating manager daemon y 2024-09-11T07:57:47.256 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:46 smithi110 ceph-mon[31709]: osdmap e71: 8 total, 8 up, 8 in 2024-09-11T07:57:47.256 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:46 smithi110 ceph-mon[31709]: mgrmap e18: y(active, starting, since 0.00975949s), standbys: x 2024-09-11T07:57:47.256 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:46 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:57:47.257 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:46 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:57:47.257 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:46 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:57:47.257 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:46 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T07:57:47.257 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:46 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T07:57:47.257 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:46 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:57:47.257 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:46 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:57:47.257 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:46 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:57:47.257 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:46 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:57:47.257 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:46 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:57:47.257 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:46 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:57:47.257 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:46 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:57:47.257 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:46 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:57:47.257 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:46 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T07:57:47.257 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:46 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T07:57:47.258 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:46 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T07:57:47.258 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:46 smithi110 ceph-mon[31709]: Manager daemon y is now available 2024-09-11T07:57:47.258 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:46 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:57:47.258 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:46 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:57:47.258 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:46 smithi110 ceph-mon[25392]: Active manager daemon y restarted 2024-09-11T07:57:47.258 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:46 smithi110 ceph-mon[25392]: Activating manager daemon y 2024-09-11T07:57:47.258 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:46 smithi110 ceph-mon[25392]: osdmap e71: 8 total, 8 up, 8 in 2024-09-11T07:57:47.258 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:46 smithi110 ceph-mon[25392]: mgrmap e18: y(active, starting, since 0.00975949s), standbys: x 2024-09-11T07:57:47.258 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:46 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:57:47.258 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:46 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:57:47.259 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:46 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:57:47.259 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:46 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T07:57:47.259 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:46 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T07:57:47.259 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:46 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:57:47.259 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:46 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:57:47.259 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:46 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:57:47.259 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:46 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:57:47.259 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:46 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:57:47.259 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:46 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:57:47.259 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:46 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:57:47.259 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:46 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:57:47.259 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:46 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T07:57:47.259 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:46 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T07:57:47.259 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:46 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T07:57:47.260 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:46 smithi110 ceph-mon[25392]: Manager daemon y is now available 2024-09-11T07:57:47.260 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:46 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:57:47.260 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:46 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:57:47.565 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:57:48.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:47 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi110", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:57:48.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:47 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi139", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:57:48.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:47 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd='[{"prefix": "config rm", "who": "osd/host:smithi110", "name": "osd_memory_target"}]': finished 2024-09-11T07:57:48.219 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:47 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd='[{"prefix": "config rm", "who": "osd/host:smithi139", "name": "osd_memory_target"}]': finished 2024-09-11T07:57:48.219 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:47 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:48.219 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:47 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:57:48.219 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:47 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:57:48.219 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:47 smithi110 ceph-mon[31709]: Updating smithi110:/etc/ceph/ceph.conf 2024-09-11T07:57:48.219 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:47 smithi110 ceph-mon[31709]: Updating smithi139:/etc/ceph/ceph.conf 2024-09-11T07:57:48.219 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:47 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:57:48.219 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:47 smithi110 ceph-mon[31709]: Standby manager daemon x restarted 2024-09-11T07:57:48.219 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:47 smithi110 ceph-mon[31709]: Standby manager daemon x started 2024-09-11T07:57:48.219 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:47 smithi110 ceph-mon[31709]: from='mgr.? 172.21.15.139:0/3690273554' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-11T07:57:48.219 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:47 smithi110 ceph-mon[31709]: from='mgr.? 172.21.15.139:0/3690273554' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T07:57:48.219 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:47 smithi110 ceph-mon[31709]: from='mgr.? 172.21.15.139:0/3690273554' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-11T07:57:48.219 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:47 smithi110 ceph-mon[31709]: from='mgr.? 172.21.15.139:0/3690273554' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T07:57:48.220 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:47 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T07:57:48.220 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:47 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T07:57:48.220 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:47 smithi110 ceph-mon[31709]: mgrmap e19: y(active, since 1.01681s), standbys: x 2024-09-11T07:57:48.220 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:47 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi110", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:57:48.220 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:47 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi139", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:57:48.220 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:47 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd='[{"prefix": "config rm", "who": "osd/host:smithi110", "name": "osd_memory_target"}]': finished 2024-09-11T07:57:48.220 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:47 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd='[{"prefix": "config rm", "who": "osd/host:smithi139", "name": "osd_memory_target"}]': finished 2024-09-11T07:57:48.220 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:47 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:48.220 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:47 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:57:48.220 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:47 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:57:48.220 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:47 smithi110 ceph-mon[25392]: Updating smithi110:/etc/ceph/ceph.conf 2024-09-11T07:57:48.221 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:47 smithi110 ceph-mon[25392]: Updating smithi139:/etc/ceph/ceph.conf 2024-09-11T07:57:48.221 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:47 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:57:48.221 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:47 smithi110 ceph-mon[25392]: Standby manager daemon x restarted 2024-09-11T07:57:48.221 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:47 smithi110 ceph-mon[25392]: Standby manager daemon x started 2024-09-11T07:57:48.221 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:47 smithi110 ceph-mon[25392]: from='mgr.? 172.21.15.139:0/3690273554' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-11T07:57:48.221 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:47 smithi110 ceph-mon[25392]: from='mgr.? 172.21.15.139:0/3690273554' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T07:57:48.221 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:47 smithi110 ceph-mon[25392]: from='mgr.? 172.21.15.139:0/3690273554' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-11T07:57:48.221 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:47 smithi110 ceph-mon[25392]: from='mgr.? 172.21.15.139:0/3690273554' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T07:57:48.221 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:47 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T07:57:48.221 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:47 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T07:57:48.221 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:47 smithi110 ceph-mon[25392]: mgrmap e19: y(active, since 1.01681s), standbys: x 2024-09-11T07:57:48.248 INFO:journalctl@ceph.iscsi.iscsi.a.smithi139.stdout:Sep 11 07:57:47 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-iscsi-iscsi-a[46755]: debug there is no tcmu-runner data available 2024-09-11T07:57:48.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:47 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi110", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:57:48.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:47 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi139", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:57:48.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:47 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd='[{"prefix": "config rm", "who": "osd/host:smithi110", "name": "osd_memory_target"}]': finished 2024-09-11T07:57:48.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:47 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd='[{"prefix": "config rm", "who": "osd/host:smithi139", "name": "osd_memory_target"}]': finished 2024-09-11T07:57:48.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:47 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:48.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:47 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:57:48.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:47 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:57:48.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:47 smithi139 ceph-mon[27311]: Updating smithi110:/etc/ceph/ceph.conf 2024-09-11T07:57:48.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:47 smithi139 ceph-mon[27311]: Updating smithi139:/etc/ceph/ceph.conf 2024-09-11T07:57:48.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:47 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:57:48.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:47 smithi139 ceph-mon[27311]: Standby manager daemon x restarted 2024-09-11T07:57:48.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:47 smithi139 ceph-mon[27311]: Standby manager daemon x started 2024-09-11T07:57:48.250 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:47 smithi139 ceph-mon[27311]: from='mgr.? 172.21.15.139:0/3690273554' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-11T07:57:48.250 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:47 smithi139 ceph-mon[27311]: from='mgr.? 172.21.15.139:0/3690273554' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T07:57:48.250 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:47 smithi139 ceph-mon[27311]: from='mgr.? 172.21.15.139:0/3690273554' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-11T07:57:48.250 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:47 smithi139 ceph-mon[27311]: from='mgr.? 172.21.15.139:0/3690273554' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T07:57:48.250 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:47 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T07:57:48.250 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:47 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T07:57:48.250 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:47 smithi139 ceph-mon[27311]: mgrmap e19: y(active, since 1.01681s), standbys: x 2024-09-11T07:57:49.405 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:49 smithi110 ceph-mon[25392]: Updating smithi139:/var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:57:49.405 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:49 smithi110 ceph-mon[25392]: Updating smithi110:/var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:57:49.405 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:49 smithi110 ceph-mon[25392]: Updating smithi139:/etc/ceph/ceph.client.admin.keyring 2024-09-11T07:57:49.405 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:49 smithi110 ceph-mon[25392]: Updating smithi110:/etc/ceph/ceph.client.admin.keyring 2024-09-11T07:57:49.405 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:49 smithi110 ceph-mon[25392]: from='client.24449 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:57:49.405 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:49 smithi110 ceph-mon[25392]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:57:49.405 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:49 smithi110 ceph-mon[25392]: Updating smithi139:/var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T07:57:49.405 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:49 smithi110 ceph-mon[25392]: Updating smithi110:/var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T07:57:49.406 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:49 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:57:49.406 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:49 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:57:49.406 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:49 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:57:49.406 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:49 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:57:49.406 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:49 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:57:49.406 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:49 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:57:49.406 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:49 smithi110 ceph-mon[25392]: Deploying daemon alertmanager.a on smithi110 2024-09-11T07:57:49.406 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:49 smithi110 ceph-mon[31709]: Updating smithi139:/var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:57:49.406 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:49 smithi110 ceph-mon[31709]: Updating smithi110:/var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:57:49.406 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:49 smithi110 ceph-mon[31709]: Updating smithi139:/etc/ceph/ceph.client.admin.keyring 2024-09-11T07:57:49.406 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:49 smithi110 ceph-mon[31709]: Updating smithi110:/etc/ceph/ceph.client.admin.keyring 2024-09-11T07:57:49.407 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:49 smithi110 ceph-mon[31709]: from='client.24449 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:57:49.407 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:49 smithi110 ceph-mon[31709]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:57:49.407 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:49 smithi110 ceph-mon[31709]: Updating smithi139:/var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T07:57:49.407 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:49 smithi110 ceph-mon[31709]: Updating smithi110:/var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T07:57:49.407 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:49 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:57:49.407 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:49 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:57:49.407 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:49 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:57:49.407 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:49 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:57:49.407 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:49 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:57:49.407 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:49 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:57:49.407 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:49 smithi110 ceph-mon[31709]: Deploying daemon alertmanager.a on smithi110 2024-09-11T07:57:49.686 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:57:49 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:57:49.403+0000 7f85f7d93640 -1 log_channel(cephadm) log [ERR] : [11/Sep/2024:07:57:49] ENGINE Error in HTTPServer.serve 2024-09-11T07:57:49.686 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:57:49 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: Traceback (most recent call last): 2024-09-11T07:57:49.686 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:57:49 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T07:57:49.686 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:57:49 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: self._connections.run(self.expiration_interval) 2024-09-11T07:57:49.686 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:57:49 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T07:57:49.686 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:57:49 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: self._run(expiration_interval) 2024-09-11T07:57:49.686 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:57:49 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T07:57:49.686 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:57:49 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T07:57:49.687 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:57:49 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T07:57:49.687 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:57:49 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T07:57:49.687 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:57:49 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T07:57:49.687 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:57:49 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: s = self.context.wrap_socket( 2024-09-11T07:57:49.687 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:57:49 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T07:57:49.687 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:57:49 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: return self.sslsocket_class._create( 2024-09-11T07:57:49.687 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:57:49 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T07:57:49.687 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:57:49 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: self.do_handshake() 2024-09-11T07:57:49.687 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:57:49 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T07:57:49.687 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:57:49 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: self._sslobj.do_handshake() 2024-09-11T07:57:49.687 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:57:49 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T07:57:49.687 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:57:49 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:57:49.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:49 smithi139 ceph-mon[27311]: Updating smithi139:/var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:57:49.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:49 smithi139 ceph-mon[27311]: Updating smithi110:/var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:57:49.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:49 smithi139 ceph-mon[27311]: Updating smithi139:/etc/ceph/ceph.client.admin.keyring 2024-09-11T07:57:49.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:49 smithi139 ceph-mon[27311]: Updating smithi110:/etc/ceph/ceph.client.admin.keyring 2024-09-11T07:57:49.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:49 smithi139 ceph-mon[27311]: from='client.24449 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:57:49.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:49 smithi139 ceph-mon[27311]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:57:49.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:49 smithi139 ceph-mon[27311]: Updating smithi139:/var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T07:57:49.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:49 smithi139 ceph-mon[27311]: Updating smithi110:/var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T07:57:49.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:49 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:57:49.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:49 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:57:49.750 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:49 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:57:49.750 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:49 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:57:49.750 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:49 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:57:49.750 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:49 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:57:49.750 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:49 smithi139 ceph-mon[27311]: Deploying daemon alertmanager.a on smithi110 2024-09-11T07:57:50.212 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T07:57:50.212 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":71,"fsid":"41bd1096-7012-11ef-bcea-c7b262605968","created":"2024-09-11T07:51:48.770375+0000","modified":"2024-09-11T07:57:46.936263+0000","last_up_change":"2024-09-11T07:56:50.396037+0000","last_in_change":"2024-09-11T07:56:31.845597+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T07:54:50.418595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-09-11T07:57:02.758662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":3,"pool_name":"datapool","create_time":"2024-09-11T07:57:02.991039+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":65,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-11T07:57:04.775543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-11T07:57:06.736275+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-11T07:57:08.802758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}}}],"osds":[{"osd":0,"uuid":"05f2cdd1-77dc-4b26-afa8-16099c528c10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":2716989082},{"type":"v1","addr":"172.21.15.110:6803","nonce":2716989082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":2716989082},{"type":"v1","addr":"172.21.15.110:6805","nonce":2716989082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":2716989082},{"type":"v1","addr":"172.21.15.110:6809","nonce":2716989082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":2716989082},{"type":"v1","addr":"172.21.15.110:6807","nonce":2716989082}]},"public_addr":"172.21.15.110:6803/2716989082","cluster_addr":"172.21.15.110:6805/2716989082","heartbeat_back_addr":"172.21.15.110:6809/2716989082","heartbeat_front_addr":"172.21.15.110:6807/2716989082","state":["exists","up"]},{"osd":1,"uuid":"1e01c286-b92c-46b3-b61c-f8aea652b8be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":1928520079},{"type":"v1","addr":"172.21.15.110:6811","nonce":1928520079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":1928520079},{"type":"v1","addr":"172.21.15.110:6813","nonce":1928520079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":1928520079},{"type":"v1","addr":"172.21.15.110:6817","nonce":1928520079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":1928520079},{"type":"v1","addr":"172.21.15.110:6815","nonce":1928520079}]},"public_addr":"172.21.15.110:6811/1928520079","cluster_addr":"172.21.15.110:6813/1928520079","heartbeat_back_addr":"172.21.15.110:6817/1928520079","heartbeat_front_addr":"172.21.15.110:6815/1928520079","state":["exists","up"]},{"osd":2,"uuid":"b413fbb5-f09d-4a04-a72d-1cc5d554f90d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":3894315382},{"type":"v1","addr":"172.21.15.110:6819","nonce":3894315382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":3894315382},{"type":"v1","addr":"172.21.15.110:6821","nonce":3894315382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":3894315382},{"type":"v1","addr":"172.21.15.110:6825","nonce":3894315382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":3894315382},{"type":"v1","addr":"172.21.15.110:6823","nonce":3894315382}]},"public_addr":"172.21.15.110:6819/3894315382","cluster_addr":"172.21.15.110:6821/3894315382","heartbeat_back_addr":"172.21.15.110:6825/3894315382","heartbeat_front_addr":"172.21.15.110:6823/3894315382","state":["exists","up"]},{"osd":3,"uuid":"293d5b4d-11ef-4d6a-9b65-5a6b47472c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":1640229305},{"type":"v1","addr":"172.21.15.110:6827","nonce":1640229305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":1640229305},{"type":"v1","addr":"172.21.15.110:6829","nonce":1640229305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":1640229305},{"type":"v1","addr":"172.21.15.110:6833","nonce":1640229305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":1640229305},{"type":"v1","addr":"172.21.15.110:6831","nonce":1640229305}]},"public_addr":"172.21.15.110:6827/1640229305","cluster_addr":"172.21.15.110:6829/1640229305","heartbeat_back_addr":"172.21.15.110:6833/1640229305","heartbeat_front_addr":"172.21.15.110:6831/1640229305","state":["exists","up"]},{"osd":4,"uuid":"f9119e96-9863-45b7-b2d5-89bc3eeb1fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2399332353},{"type":"v1","addr":"172.21.15.139:6801","nonce":2399332353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2399332353},{"type":"v1","addr":"172.21.15.139:6803","nonce":2399332353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2399332353},{"type":"v1","addr":"172.21.15.139:6807","nonce":2399332353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2399332353},{"type":"v1","addr":"172.21.15.139:6805","nonce":2399332353}]},"public_addr":"172.21.15.139:6801/2399332353","cluster_addr":"172.21.15.139:6803/2399332353","heartbeat_back_addr":"172.21.15.139:6807/2399332353","heartbeat_front_addr":"172.21.15.139:6805/2399332353","state":["exists","up"]},{"osd":5,"uuid":"3339eb92-7017-4bb8-947f-557a32723e94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1641418305},{"type":"v1","addr":"172.21.15.139:6809","nonce":1641418305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1641418305},{"type":"v1","addr":"172.21.15.139:6811","nonce":1641418305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1641418305},{"type":"v1","addr":"172.21.15.139:6815","nonce":1641418305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1641418305},{"type":"v1","addr":"172.21.15.139:6813","nonce":1641418305}]},"public_addr":"172.21.15.139:6809/1641418305","cluster_addr":"172.21.15.139:6811/1641418305","heartbeat_back_addr":"172.21.15.139:6815/1641418305","heartbeat_front_addr":"172.21.15.139:6813/1641418305","state":["exists","up"]},{"osd":6,"uuid":"69a4457c-39cc-41f5-bceb-9229b8b030f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1783109268},{"type":"v1","addr":"172.21.15.139:6817","nonce":1783109268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1783109268},{"type":"v1","addr":"172.21.15.139:6819","nonce":1783109268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1783109268},{"type":"v1","addr":"172.21.15.139:6823","nonce":1783109268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1783109268},{"type":"v1","addr":"172.21.15.139:6821","nonce":1783109268}]},"public_addr":"172.21.15.139:6817/1783109268","cluster_addr":"172.21.15.139:6819/1783109268","heartbeat_back_addr":"172.21.15.139:6823/1783109268","heartbeat_front_addr":"172.21.15.139:6821/1783109268","state":["exists","up"]},{"osd":7,"uuid":"c58219c0-3caa-4b99-afb1-bc572ac152ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":232507052},{"type":"v1","addr":"172.21.15.139:6825","nonce":232507052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":232507052},{"type":"v1","addr":"172.21.15.139:6827","nonce":232507052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":232507052},{"type":"v1","addr":"172.21.15.139:6831","nonce":232507052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":232507052},{"type":"v1","addr":"172.21.15.139:6829","nonce":232507052}]},"public_addr":"172.21.15.139:6825/232507052","cluster_addr":"172.21.15.139:6827/232507052","heartbeat_back_addr":"172.21.15.139:6831/232507052","heartbeat_front_addr":"172.21.15.139:6829/232507052","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:53:57.309408+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:54:22.407580+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:54:47.670630+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:55:13.231294+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:55:37.035454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:56:00.079531+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:56:24.721942+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:56:48.981334+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":7,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.110:0/1006666897":"2024-09-12T07:57:46.936218+0000","172.21.15.110:0/1550053204":"2024-09-12T07:57:46.936218+0000","172.21.15.110:6800/1242491477":"2024-09-12T07:52:10.347528+0000","172.21.15.110:0/828045681":"2024-09-12T07:52:10.347528+0000","172.21.15.110:0/2224974349":"2024-09-12T07:57:46.936218+0000","172.21.15.110:0/1589810572":"2024-09-12T07:52:10.347528+0000","172.21.15.110:0/4293537776":"2024-09-12T07:57:46.936218+0000","172.21.15.110:0/2727007179":"2024-09-12T07:52:10.347528+0000","172.21.15.110:6801/1242491477":"2024-09-12T07:52:10.347528+0000","172.21.15.110:6800/4096534802":"2024-09-12T07:52:32.111455+0000","172.21.15.110:0/3707065533":"2024-09-12T07:52:32.111455+0000","172.21.15.110:0/1603882420":"2024-09-12T07:52:32.111455+0000","172.21.15.110:6801/4096534802":"2024-09-12T07:52:32.111455+0000","172.21.15.110:0/2391676173":"2024-09-12T07:52:32.111455+0000","172.21.15.110:6800/248793870":"2024-09-12T07:57:46.936218+0000","172.21.15.110:0/3473535153":"2024-09-12T07:57:46.936218+0000","172.21.15.110:6801/248793870":"2024-09-12T07:57:46.936218+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T07:57:50.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:50 smithi110 ceph-mon[25392]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:57:50.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:50 smithi110 ceph-mon[25392]: [11/Sep/2024:07:57:49] ENGINE Bus STARTING 2024-09-11T07:57:50.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:50 smithi110 ceph-mon[25392]: mgrmap e20: y(active, since 2s), standbys: x 2024-09-11T07:57:50.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:50 smithi110 ceph-mon[25392]: [11/Sep/2024:07:57:49] ENGINE Serving on https://172.21.15.110:7150 2024-09-11T07:57:50.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:50 smithi110 ceph-mon[25392]: [11/Sep/2024:07:57:49] ENGINE Bus STARTED 2024-09-11T07:57:50.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:50 smithi110 ceph-mon[25392]: [11/Sep/2024:07:57:49] ENGINE Error in HTTPServer.serve 2024-09-11T07:57:50.468 INFO:journalctl@ceph.mon.a.smithi110.stdout: Traceback (most recent call last): 2024-09-11T07:57:50.469 INFO:journalctl@ceph.mon.a.smithi110.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T07:57:50.469 INFO:journalctl@ceph.mon.a.smithi110.stdout: self._connections.run(self.expiration_interval) 2024-09-11T07:57:50.469 INFO:journalctl@ceph.mon.a.smithi110.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T07:57:50.469 INFO:journalctl@ceph.mon.a.smithi110.stdout: self._run(expiration_interval) 2024-09-11T07:57:50.469 INFO:journalctl@ceph.mon.a.smithi110.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T07:57:50.469 INFO:journalctl@ceph.mon.a.smithi110.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T07:57:50.469 INFO:journalctl@ceph.mon.a.smithi110.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T07:57:50.469 INFO:journalctl@ceph.mon.a.smithi110.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T07:57:50.469 INFO:journalctl@ceph.mon.a.smithi110.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T07:57:50.469 INFO:journalctl@ceph.mon.a.smithi110.stdout: s = self.context.wrap_socket( 2024-09-11T07:57:50.469 INFO:journalctl@ceph.mon.a.smithi110.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T07:57:50.469 INFO:journalctl@ceph.mon.a.smithi110.stdout: return self.sslsocket_class._create( 2024-09-11T07:57:50.469 INFO:journalctl@ceph.mon.a.smithi110.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T07:57:50.469 INFO:journalctl@ceph.mon.a.smithi110.stdout: self.do_handshake() 2024-09-11T07:57:50.470 INFO:journalctl@ceph.mon.a.smithi110.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T07:57:50.470 INFO:journalctl@ceph.mon.a.smithi110.stdout: self._sslobj.do_handshake() 2024-09-11T07:57:50.470 INFO:journalctl@ceph.mon.a.smithi110.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T07:57:50.470 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:50 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:57:50.470 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:50 smithi110 ceph-mon[25392]: from='client.? 172.21.15.110:0/1806529677' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T07:57:50.470 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:50 smithi110 ceph-mon[31709]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:57:50.470 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:50 smithi110 ceph-mon[31709]: [11/Sep/2024:07:57:49] ENGINE Bus STARTING 2024-09-11T07:57:50.470 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:50 smithi110 ceph-mon[31709]: mgrmap e20: y(active, since 2s), standbys: x 2024-09-11T07:57:50.470 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:50 smithi110 ceph-mon[31709]: [11/Sep/2024:07:57:49] ENGINE Serving on https://172.21.15.110:7150 2024-09-11T07:57:50.470 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:50 smithi110 ceph-mon[31709]: [11/Sep/2024:07:57:49] ENGINE Bus STARTED 2024-09-11T07:57:50.470 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:50 smithi110 ceph-mon[31709]: [11/Sep/2024:07:57:49] ENGINE Error in HTTPServer.serve 2024-09-11T07:57:50.470 INFO:journalctl@ceph.mon.c.smithi110.stdout: Traceback (most recent call last): 2024-09-11T07:57:50.471 INFO:journalctl@ceph.mon.c.smithi110.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T07:57:50.471 INFO:journalctl@ceph.mon.c.smithi110.stdout: self._connections.run(self.expiration_interval) 2024-09-11T07:57:50.471 INFO:journalctl@ceph.mon.c.smithi110.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T07:57:50.471 INFO:journalctl@ceph.mon.c.smithi110.stdout: self._run(expiration_interval) 2024-09-11T07:57:50.471 INFO:journalctl@ceph.mon.c.smithi110.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T07:57:50.471 INFO:journalctl@ceph.mon.c.smithi110.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T07:57:50.471 INFO:journalctl@ceph.mon.c.smithi110.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T07:57:50.471 INFO:journalctl@ceph.mon.c.smithi110.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T07:57:50.471 INFO:journalctl@ceph.mon.c.smithi110.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T07:57:50.471 INFO:journalctl@ceph.mon.c.smithi110.stdout: s = self.context.wrap_socket( 2024-09-11T07:57:50.471 INFO:journalctl@ceph.mon.c.smithi110.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T07:57:50.471 INFO:journalctl@ceph.mon.c.smithi110.stdout: return self.sslsocket_class._create( 2024-09-11T07:57:50.471 INFO:journalctl@ceph.mon.c.smithi110.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T07:57:50.471 INFO:journalctl@ceph.mon.c.smithi110.stdout: self.do_handshake() 2024-09-11T07:57:50.472 INFO:journalctl@ceph.mon.c.smithi110.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T07:57:50.472 INFO:journalctl@ceph.mon.c.smithi110.stdout: self._sslobj.do_handshake() 2024-09-11T07:57:50.472 INFO:journalctl@ceph.mon.c.smithi110.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T07:57:50.472 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:50 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:57:50.472 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:50 smithi110 ceph-mon[31709]: from='client.? 172.21.15.110:0/1806529677' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T07:57:50.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:50 smithi139 ceph-mon[27311]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:57:50.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:50 smithi139 ceph-mon[27311]: [11/Sep/2024:07:57:49] ENGINE Bus STARTING 2024-09-11T07:57:50.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:50 smithi139 ceph-mon[27311]: mgrmap e20: y(active, since 2s), standbys: x 2024-09-11T07:57:50.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:50 smithi139 ceph-mon[27311]: [11/Sep/2024:07:57:49] ENGINE Serving on https://172.21.15.110:7150 2024-09-11T07:57:50.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:50 smithi139 ceph-mon[27311]: [11/Sep/2024:07:57:49] ENGINE Bus STARTED 2024-09-11T07:57:50.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:50 smithi139 ceph-mon[27311]: [11/Sep/2024:07:57:49] ENGINE Error in HTTPServer.serve 2024-09-11T07:57:50.749 INFO:journalctl@ceph.mon.b.smithi139.stdout: Traceback (most recent call last): 2024-09-11T07:57:50.749 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T07:57:50.749 INFO:journalctl@ceph.mon.b.smithi139.stdout: self._connections.run(self.expiration_interval) 2024-09-11T07:57:50.749 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T07:57:50.750 INFO:journalctl@ceph.mon.b.smithi139.stdout: self._run(expiration_interval) 2024-09-11T07:57:50.750 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T07:57:50.750 INFO:journalctl@ceph.mon.b.smithi139.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T07:57:50.750 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T07:57:50.750 INFO:journalctl@ceph.mon.b.smithi139.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T07:57:50.750 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T07:57:50.750 INFO:journalctl@ceph.mon.b.smithi139.stdout: s = self.context.wrap_socket( 2024-09-11T07:57:50.750 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T07:57:50.750 INFO:journalctl@ceph.mon.b.smithi139.stdout: return self.sslsocket_class._create( 2024-09-11T07:57:50.750 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T07:57:50.750 INFO:journalctl@ceph.mon.b.smithi139.stdout: self.do_handshake() 2024-09-11T07:57:50.750 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T07:57:50.751 INFO:journalctl@ceph.mon.b.smithi139.stdout: self._sslobj.do_handshake() 2024-09-11T07:57:50.751 INFO:journalctl@ceph.mon.b.smithi139.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T07:57:50.751 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:50 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:57:50.751 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:50 smithi139 ceph-mon[27311]: from='client.? 172.21.15.110:0/1806529677' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T07:57:51.043 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-11T07:57:51.043 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-11T07:57:51.044 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-11T07:57:51.044 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-11T07:57:51.045 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-11T07:57:51.045 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-11T07:57:51.046 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-11T07:57:51.046 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-11T07:57:51.524 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:57:51.565 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:57:52.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:52 smithi110 ceph-mon[25392]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:57:52.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:52 smithi110 ceph-mon[25392]: mgrmap e21: y(active, since 4s), standbys: x 2024-09-11T07:57:52.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:52 smithi110 ceph-mon[31709]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:57:52.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:52 smithi110 ceph-mon[31709]: mgrmap e21: y(active, since 4s), standbys: x 2024-09-11T07:57:52.748 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:52 smithi139 ceph-mon[27311]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:57:52.748 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:52 smithi139 ceph-mon[27311]: mgrmap e21: y(active, since 4s), standbys: x 2024-09-11T07:57:53.204 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:57:53.204 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:57:53.205 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:57:53.205 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:57:53.697 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:57:53.698 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:57:54.248 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:57:53 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x[29520]: ::ffff:172.21.15.139 - - [11/Sep/2024:07:57:53] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:57:54.653 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:54 smithi110 ceph-mon[31709]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1 KiB/s rd, 1 op/s 2024-09-11T07:57:54.653 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:54 smithi110 ceph-mon[25392]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1 KiB/s rd, 1 op/s 2024-09-11T07:57:54.748 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:54 smithi139 ceph-mon[27311]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1 KiB/s rd, 1 op/s 2024-09-11T07:57:56.536 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:56 smithi110 ceph-mon[31709]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-11T07:57:56.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:56 smithi110 ceph-mon[25392]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-11T07:57:56.748 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:56 smithi139 ceph-mon[27311]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-11T07:57:58.248 INFO:journalctl@ceph.iscsi.iscsi.a.smithi139.stdout:Sep 11 07:57:57 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-iscsi-iscsi-a[46755]: debug there is no tcmu-runner data available 2024-09-11T07:57:58.542 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:58 smithi110 ceph-mon[31709]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-11T07:57:58.542 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:58 smithi110 ceph-mon[25392]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-11T07:57:58.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:58 smithi139 ceph-mon[27311]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-11T07:57:59.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:57:59 smithi139 ceph-mon[27311]: from='client.24449 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:57:59.698 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:57:59 smithi110 ceph-mon[31709]: from='client.24449 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:57:59.699 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:57:59 smithi110 ceph-mon[25392]: from='client.24449 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:58:00.534 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:00 smithi139 ceph-mon[27311]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-11T07:58:00.689 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:00 smithi110 ceph-mon[25392]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-11T07:58:00.690 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:00 smithi110 ceph-mon[31709]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-11T07:58:02.498 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:02 smithi139 ceph-mon[27311]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-11T07:58:02.498 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:02 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:58:02.498 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:02 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:02.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:02 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:02.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:02 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:02.654 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:02 smithi110 ceph-mon[25392]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-11T07:58:02.654 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:02 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:58:02.654 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:02 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:02.654 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:02 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:02.654 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:02 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:02.654 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:02 smithi110 ceph-mon[31709]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-11T07:58:02.654 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:02 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:58:02.654 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:02 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:02.655 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:02 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:02.655 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:02 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:04.248 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:58:03 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x[29520]: ::ffff:172.21.15.139 - - [11/Sep/2024:07:58:03] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:58:04.592 INFO:teuthology.orchestra.run.smithi110.stdout:55834574893 2024-09-11T07:58:04.592 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-11T07:58:04.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:04 smithi110 ceph-mon[31709]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-11T07:58:04.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:04 smithi110 ceph-mon[25392]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-11T07:58:04.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:04 smithi139 ceph-mon[27311]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-11T07:58:05.974 INFO:teuthology.orchestra.run.smithi110.stdout:163208757274 2024-09-11T07:58:05.975 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-11T07:58:06.561 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:06 smithi110 ceph-mon[31709]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-11T07:58:06.562 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:06 smithi110 ceph-mon[25392]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-11T07:58:06.748 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:06 smithi139 ceph-mon[27311]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-11T07:58:07.381 INFO:teuthology.orchestra.run.smithi110.stdout:128849018911 2024-09-11T07:58:07.381 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-11T07:58:08.248 INFO:journalctl@ceph.iscsi.iscsi.a.smithi139.stdout:Sep 11 07:58:07 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-iscsi-iscsi-a[46755]: debug there is no tcmu-runner data available 2024-09-11T07:58:08.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:08 smithi110 ceph-mon[25392]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:58:08.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:08 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:08.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:08 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:08.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:08 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:08.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:08 smithi110 ceph-mon[31709]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:58:08.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:08 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:08.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:08 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:08.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:08 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:08.748 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:08 smithi139 ceph-mon[27311]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:58:08.748 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:08 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:08.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:08 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:08.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:08 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:09.111 INFO:teuthology.orchestra.run.smithi110.stdout:197568495638 2024-09-11T07:58:09.111 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-11T07:58:09.641 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:09 smithi110 ceph-mon[25392]: from='client.24449 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:58:09.641 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:09 smithi110 ceph-mon[31709]: from='client.24449 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:58:09.748 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:09 smithi139 ceph-mon[27311]: from='client.24449 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:58:10.699 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:10 smithi110 ceph-mon[25392]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:58:10.699 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:10 smithi110 ceph-mon[31709]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:58:10.722 INFO:teuthology.orchestra.run.smithi110.stdout:34359738420 2024-09-11T07:58:10.722 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-11T07:58:10.748 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:10 smithi139 ceph-mon[27311]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:58:12.147 INFO:teuthology.orchestra.run.smithi110.stdout:77309411371 2024-09-11T07:58:12.147 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-11T07:58:12.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:12 smithi110 ceph-mon[25392]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:58:12.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:12 smithi110 ceph-mon[31709]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:58:12.748 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:12 smithi139 ceph-mon[27311]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:58:12.803 INFO:teuthology.orchestra.run.smithi110.stdout:231928234002 2024-09-11T07:58:12.803 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-11T07:58:13.799 INFO:teuthology.orchestra.run.smithi110.stdout:107374182438 2024-09-11T07:58:13.800 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-11T07:58:13.999 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:58:13 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x[29520]: ::ffff:172.21.15.139 - - [11/Sep/2024:07:58:13] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:58:14.620 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:58:14.630 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:58:14.633 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:58:14.641 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:58:14.670 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:58:14.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:14 smithi110 ceph-mon[25392]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:58:14.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:14 smithi110 ceph-mon[31709]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:58:14.719 INFO:journalctl@ceph.alertmanager.a.smithi110.stdout:Sep 11 07:58:14 smithi110 systemd[1]: Started Ceph alertmanager.a for 41bd1096-7012-11ef-bcea-c7b262605968. 2024-09-11T07:58:14.748 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:14 smithi139 ceph-mon[27311]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:58:15.057 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:58:14 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: [11/Sep/2024:07:58:14] ENGINE Bus STOPPING 2024-09-11T07:58:15.313 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:58:15 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: [11/Sep/2024:07:58:15] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-11T07:58:15.313 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:58:15 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: [11/Sep/2024:07:58:15] ENGINE Bus STOPPED 2024-09-11T07:58:15.313 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:58:15 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: [11/Sep/2024:07:58:15] ENGINE Bus STARTING 2024-09-11T07:58:15.608 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:58:15 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: [11/Sep/2024:07:58:15] ENGINE Serving on http://:::9283 2024-09-11T07:58:15.608 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:58:15 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: [11/Sep/2024:07:58:15] ENGINE Bus STARTED 2024-09-11T07:58:15.648 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:58:15.648 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:58:15.868 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:15 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:15.868 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:15 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:15.868 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:15 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:15.868 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:15 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:15.868 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:15 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:15.869 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:15 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:15.869 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:15 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-11T07:58:15.869 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:15 smithi110 ceph-mon[31709]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-11T07:58:15.869 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:15 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:15.869 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:15 smithi110 ceph-mon[31709]: Deploying daemon grafana.a on smithi139 2024-09-11T07:58:15.870 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:15 smithi110 ceph-mon[31709]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:58:15.870 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:15 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:15.870 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:15 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:15.870 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:15 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:15.870 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:15 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:15.871 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:15 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:15.871 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:15 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:15.871 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:15 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-11T07:58:15.871 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:15 smithi110 ceph-mon[25392]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-11T07:58:15.871 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:15 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:15.871 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:15 smithi110 ceph-mon[25392]: Deploying daemon grafana.a on smithi139 2024-09-11T07:58:15.872 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:15 smithi110 ceph-mon[25392]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:58:15.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:15 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:15.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:15 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:15.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:15 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:15.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:15 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:15.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:15 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:15.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:15 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:15.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:15 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-11T07:58:15.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:15 smithi139 ceph-mon[27311]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-11T07:58:15.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:15 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:15.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:15 smithi139 ceph-mon[27311]: Deploying daemon grafana.a on smithi139 2024-09-11T07:58:15.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:15 smithi139 ceph-mon[27311]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:58:16.718 INFO:journalctl@ceph.alertmanager.a.smithi110.stdout:Sep 11 07:58:16 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-alertmanager-a[56448]: ts=2024-09-11T07:58:16.472Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000506084s 2024-09-11T07:58:18.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:18 smithi110 ceph-mon[25392]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:58:18.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:18 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:58:18.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:18 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:18.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:18 smithi110 ceph-mon[31709]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:58:18.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:18 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:58:18.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:18 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:18.248 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:18 smithi139 ceph-mon[27311]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:58:18.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:18 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:58:18.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:18 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:18.249 INFO:journalctl@ceph.iscsi.iscsi.a.smithi139.stdout:Sep 11 07:58:17 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-iscsi-iscsi-a[46755]: debug there is no tcmu-runner data available 2024-09-11T07:58:19.248 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:19 smithi139 ceph-mon[27311]: from='client.24449 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:58:19.347 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:19 smithi110 ceph-mon[31709]: from='client.24449 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:58:19.347 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:19 smithi110 ceph-mon[25392]: from='client.24449 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:58:20.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:20 smithi110 ceph-mon[25392]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:58:20.287 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:20 smithi110 ceph-mon[31709]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:58:20.498 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:20 smithi139 ceph-mon[27311]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:58:20.596 INFO:teuthology.orchestra.run.smithi110.stdout:197568495640 2024-09-11T07:58:21.303 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:21 smithi110 ceph-mon[31709]: from='client.? 172.21.15.110:0/1467632649' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-11T07:58:21.303 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:21 smithi110 ceph-mon[25392]: from='client.? 172.21.15.110:0/1467632649' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-11T07:58:21.498 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:21 smithi139 ceph-mon[27311]: from='client.? 172.21.15.110:0/1467632649' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-11T07:58:22.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:22 smithi110 ceph-mon[25392]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:58:22.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:22 smithi110 ceph-mon[31709]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:58:22.248 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:22 smithi139 ceph-mon[27311]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:58:22.721 INFO:teuthology.orchestra.run.smithi110.stdout:34359738422 2024-09-11T07:58:23.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:23 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:23.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:23 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:23.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:23 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:23.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:23 smithi139 ceph-mon[27311]: from='client.? 172.21.15.110:0/3245155599' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-11T07:58:23.665 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:23 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:23.665 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:23 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:23.665 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:23 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:23.665 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:23 smithi110 ceph-mon[25392]: from='client.? 172.21.15.110:0/3245155599' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-11T07:58:23.666 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:23 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:23.666 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:23 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:23.666 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:23 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:23.666 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:23 smithi110 ceph-mon[31709]: from='client.? 172.21.15.110:0/3245155599' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-11T07:58:23.954 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:58:23 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x[29520]: ::ffff:172.21.15.139 - - [11/Sep/2024:07:58:23] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:58:24.468 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:24 smithi110 ceph-mon[31709]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:58:24.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:24 smithi110 ceph-mon[25392]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:58:24.549 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495638 got 197568495640 for osd.6 2024-09-11T07:58:24.549 DEBUG:teuthology.parallel:result is None 2024-09-11T07:58:24.747 INFO:journalctl@ceph.alertmanager.a.smithi110.stdout:Sep 11 07:58:24 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-alertmanager-a[56448]: ts=2024-09-11T07:58:24.474Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.002126876s 2024-09-11T07:58:24.748 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:24 smithi139 ceph-mon[27311]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:58:25.142 INFO:teuthology.orchestra.run.smithi110.stdout:55834574898 2024-09-11T07:58:25.354 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 systemd[1]: Starting Ceph grafana.a for 41bd1096-7012-11ef-bcea-c7b262605968... 2024-09-11T07:58:25.451 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:25 smithi110 ceph-mon[25392]: from='client.? 172.21.15.110:0/1590300583' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-11T07:58:25.451 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:25 smithi110 ceph-mon[31709]: from='client.? 172.21.15.110:0/1590300583' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-11T07:58:25.623 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-mon[27311]: from='client.? 172.21.15.110:0/1590300583' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-11T07:58:25.623 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 podman[51021]: 2024-09-11 07:58:25.302043035 +0000 UTC m=+0.015348403 image pull 954c08fa618887fe9a6fe572ae541197c77e625b05f4193df7898f5ac649ada9 quay.io/ceph/ceph-grafana:9.4.7 2024-09-11T07:58:25.624 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 podman[51021]: 2024-09-11 07:58:25.412624514 +0000 UTC m=+0.125929871 container create 551749459dfbadccadfbb90889faa9cfb1667d54f8f346406d3852108b42971a (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a, vendor=Red Hat, Inc., vcs-type=git, maintainer=Paul Cuzner , release=239.1651231664, io.k8s.display-name=Red Hat Universal Base Image 8, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, description=Ceph Grafana Container, io.buildah.version=1.32.0, name=ubi8, version=8.5, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.openshift.tags=base rhel8, architecture=x86_64, build-date=2022-04-29T11:48:11.055897, summary=Grafana Container configured for Ceph mgr/dashboard integration, com.redhat.component=ubi8-container, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, io.openshift.expose-services=, distribution-scope=public) 2024-09-11T07:58:25.624 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 podman[51021]: 2024-09-11 07:58:25.622921956 +0000 UTC m=+0.336227315 container init 551749459dfbadccadfbb90889faa9cfb1667d54f8f346406d3852108b42971a (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, vendor=Red Hat, Inc., distribution-scope=public, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, description=Ceph Grafana Container, release=239.1651231664, io.k8s.display-name=Red Hat Universal Base Image 8, io.openshift.tags=base rhel8, architecture=x86_64, maintainer=Paul Cuzner , url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, io.buildah.version=1.32.0, io.openshift.expose-services=, build-date=2022-04-29T11:48:11.055897, version=8.5, vcs-type=git, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., com.redhat.component=ubi8-container, summary=Grafana Container configured for Ceph mgr/dashboard integration, name=ubi8) 2024-09-11T07:58:25.875 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 podman[51021]: 2024-09-11 07:58:25.62595106 +0000 UTC m=+0.339256422 container start 551749459dfbadccadfbb90889faa9cfb1667d54f8f346406d3852108b42971a (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a, vendor=Red Hat, Inc., maintainer=Paul Cuzner , vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, io.openshift.expose-services=, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, build-date=2022-04-29T11:48:11.055897, io.buildah.version=1.32.0, name=ubi8, io.openshift.tags=base rhel8, vcs-type=git, architecture=x86_64, distribution-scope=public, io.k8s.display-name=Red Hat Universal Base Image 8, description=Ceph Grafana Container, release=239.1651231664, summary=Grafana Container configured for Ceph mgr/dashboard integration, version=8.5, com.redhat.component=ubi8-container, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664) 2024-09-11T07:58:25.876 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 bash[51021]: 551749459dfbadccadfbb90889faa9cfb1667d54f8f346406d3852108b42971a 2024-09-11T07:58:25.876 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 systemd[1]: Started Ceph grafana.a for 41bd1096-7012-11ef-bcea-c7b262605968. 2024-09-11T07:58:25.876 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=settings t=2024-09-11T07:58:25.779168335Z level=info msg="Starting Grafana" version=9.4.7 commit=4add91f03d branch=HEAD compiled=2023-03-16T23:56:52Z 2024-09-11T07:58:25.876 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=settings t=2024-09-11T07:58:25.779355541Z level=warn msg="\"sentry\" frontend logging provider is deprecated and will be removed in the next major version. Use \"grafana\" provider instead." 2024-09-11T07:58:25.876 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=settings t=2024-09-11T07:58:25.779374186Z level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini 2024-09-11T07:58:25.876 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=settings t=2024-09-11T07:58:25.779381458Z level=info msg="Config loaded from" file=/etc/grafana/grafana.ini 2024-09-11T07:58:25.876 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=settings t=2024-09-11T07:58:25.779387499Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana" 2024-09-11T07:58:25.876 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=settings t=2024-09-11T07:58:25.779393382Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_LOGS=/var/log/grafana" 2024-09-11T07:58:25.876 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=settings t=2024-09-11T07:58:25.779399Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-09-11T07:58:25.877 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=settings t=2024-09-11T07:58:25.779404771Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-09-11T07:58:25.877 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=settings t=2024-09-11T07:58:25.779410497Z level=info msg="Path Home" path=/usr/share/grafana 2024-09-11T07:58:25.877 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=settings t=2024-09-11T07:58:25.779416146Z level=info msg="Path Data" path=/var/lib/grafana 2024-09-11T07:58:25.877 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=settings t=2024-09-11T07:58:25.77942171Z level=info msg="Path Logs" path=/var/log/grafana 2024-09-11T07:58:25.877 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=settings t=2024-09-11T07:58:25.779427136Z level=info msg="Path Plugins" path=/var/lib/grafana/plugins 2024-09-11T07:58:25.877 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=settings t=2024-09-11T07:58:25.779432751Z level=info msg="Path Provisioning" path=/etc/grafana/provisioning 2024-09-11T07:58:25.877 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=settings t=2024-09-11T07:58:25.779438402Z level=info msg="App mode production" 2024-09-11T07:58:25.877 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=sqlstore t=2024-09-11T07:58:25.779484199Z level=info msg="Connecting to DB" dbtype=sqlite3 2024-09-11T07:58:25.877 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=sqlstore t=2024-09-11T07:58:25.77949957Z level=warn msg="SQLite database file has broader permissions than it should" path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-09-11T07:58:25.877 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.780341957Z level=info msg="Starting DB migrations" 2024-09-11T07:58:25.877 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.780420537Z level=info msg="Executing migration" id="create migration_log table" 2024-09-11T07:58:25.877 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.780952356Z level=info msg="Executing migration" id="create user table" 2024-09-11T07:58:25.877 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.78138307Z level=info msg="Executing migration" id="add unique index user.login" 2024-09-11T07:58:25.878 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.782219274Z level=info msg="Executing migration" id="add unique index user.email" 2024-09-11T07:58:25.878 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.784891997Z level=info msg="Executing migration" id="drop index UQE_user_login - v1" 2024-09-11T07:58:25.878 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.787628621Z level=info msg="Executing migration" id="drop index UQE_user_email - v1" 2024-09-11T07:58:25.878 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.790485612Z level=info msg="Executing migration" id="Rename table user to user_v1 - v1" 2024-09-11T07:58:25.878 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.793311589Z level=info msg="Executing migration" id="create user table v2" 2024-09-11T07:58:25.878 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.796044918Z level=info msg="Executing migration" id="create index UQE_user_login - v2" 2024-09-11T07:58:25.878 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.798752484Z level=info msg="Executing migration" id="create index UQE_user_email - v2" 2024-09-11T07:58:25.878 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.799265605Z level=info msg="Executing migration" id="copy data_source v1 to v2" 2024-09-11T07:58:25.878 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.802018191Z level=info msg="Executing migration" id="Drop old table user_v1" 2024-09-11T07:58:25.878 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.804829185Z level=info msg="Executing migration" id="Add column help_flags1 to user table" 2024-09-11T07:58:25.878 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.807725513Z level=info msg="Executing migration" id="Update user table charset" 2024-09-11T07:58:25.879 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.810576402Z level=info msg="Executing migration" id="Add last_seen_at column to user" 2024-09-11T07:58:25.879 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.813585075Z level=info msg="Executing migration" id="Add missing user data" 2024-09-11T07:58:25.879 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.816355853Z level=info msg="Executing migration" id="Add is_disabled column to user" 2024-09-11T07:58:25.879 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.819160088Z level=info msg="Executing migration" id="Add index user.login/user.email" 2024-09-11T07:58:25.879 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.821835234Z level=info msg="Executing migration" id="Add is_service_account column to user" 2024-09-11T07:58:25.879 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.824564931Z level=info msg="Executing migration" id="Update is_service_account column to nullable" 2024-09-11T07:58:25.879 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.827342842Z level=info msg="Executing migration" id="create temp user table v1-7" 2024-09-11T07:58:25.879 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.830137539Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v1-7" 2024-09-11T07:58:25.880 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.832814732Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v1-7" 2024-09-11T07:58:25.880 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.835410266Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v1-7" 2024-09-11T07:58:25.880 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.837985455Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v1-7" 2024-09-11T07:58:25.880 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.838443598Z level=info msg="Executing migration" id="Update temp_user table charset" 2024-09-11T07:58:25.880 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.841111183Z level=info msg="Executing migration" id="drop index IDX_temp_user_email - v1" 2024-09-11T07:58:25.880 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.841600035Z level=info msg="Executing migration" id="drop index IDX_temp_user_org_id - v1" 2024-09-11T07:58:25.880 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.843729567Z level=info msg="Executing migration" id="drop index IDX_temp_user_code - v1" 2024-09-11T07:58:25.880 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.844150751Z level=info msg="Executing migration" id="drop index IDX_temp_user_status - v1" 2024-09-11T07:58:25.880 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.844560122Z level=info msg="Executing migration" id="Rename table temp_user to temp_user_tmp_qwerty - v1" 2024-09-11T07:58:25.880 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.847185589Z level=info msg="Executing migration" id="create temp_user v2" 2024-09-11T07:58:25.880 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.849957273Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v2" 2024-09-11T07:58:25.880 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.852650774Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v2" 2024-09-11T07:58:25.880 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.85308601Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v2" 2024-09-11T07:58:25.881 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.855905688Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v2" 2024-09-11T07:58:25.881 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.858556338Z level=info msg="Executing migration" id="copy temp_user v1 to v2" 2024-09-11T07:58:25.881 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.861198254Z level=info msg="Executing migration" id="drop temp_user_tmp_qwerty" 2024-09-11T07:58:25.881 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.863858329Z level=info msg="Executing migration" id="Set created for temp users that will otherwise prematurely expire" 2024-09-11T07:58:25.881 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.864730301Z level=info msg="Executing migration" id="create star table" 2024-09-11T07:58:25.881 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.865529265Z level=info msg="Executing migration" id="add unique index star.user_id_dashboard_id" 2024-09-11T07:58:25.881 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.866486176Z level=info msg="Executing migration" id="create org table v1" 2024-09-11T07:58:25.881 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.867153158Z level=info msg="Executing migration" id="create index UQE_org_name - v1" 2024-09-11T07:58:25.881 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.868163285Z level=info msg="Executing migration" id="create org_user table v1" 2024-09-11T07:58:25.881 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.868815558Z level=info msg="Executing migration" id="create index IDX_org_user_org_id - v1" 2024-09-11T07:58:25.881 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.869636253Z level=info msg="Executing migration" id="create index UQE_org_user_org_id_user_id - v1" 2024-09-11T07:58:25.881 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.870495006Z level=info msg="Executing migration" id="create index IDX_org_user_user_id - v1" 2024-09-11T07:58:25.882 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.871469448Z level=info msg="Executing migration" id="Update org table charset" 2024-09-11T07:58:25.882 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.872187963Z level=info msg="Executing migration" id="Update org_user table charset" 2024-09-11T07:58:25.882 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.873081483Z level=info msg="Executing migration" id="Migrate all Read Only Viewers to Viewers" 2024-09-11T07:58:25.882 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.87424031Z level=info msg="Executing migration" id="create dashboard table" 2024-09-11T07:58:25.882 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.87468258Z level=info msg="Executing migration" id="add index dashboard.account_id" 2024-09-11T07:58:25.882 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.875546313Z level=info msg="Executing migration" id="add unique index dashboard_account_id_slug" 2024-09-11T07:58:25.882 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.876386272Z level=info msg="Executing migration" id="create dashboard_tag table" 2024-09-11T07:58:26.125 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.877367431Z level=info msg="Executing migration" id="add unique index dashboard_tag.dasboard_id_term" 2024-09-11T07:58:26.125 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.878066481Z level=info msg="Executing migration" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2024-09-11T07:58:26.125 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.879042833Z level=info msg="Executing migration" id="Rename table dashboard to dashboard_v1 - v1" 2024-09-11T07:58:26.125 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.879914156Z level=info msg="Executing migration" id="create dashboard v2" 2024-09-11T07:58:26.125 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.880749122Z level=info msg="Executing migration" id="create index IDX_dashboard_org_id - v2" 2024-09-11T07:58:26.126 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.881755582Z level=info msg="Executing migration" id="create index UQE_dashboard_org_id_slug - v2" 2024-09-11T07:58:26.126 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.882462017Z level=info msg="Executing migration" id="copy dashboard v1 to v2" 2024-09-11T07:58:26.126 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.883300516Z level=info msg="Executing migration" id="drop table dashboard_v1" 2024-09-11T07:58:26.126 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.884195703Z level=info msg="Executing migration" id="alter dashboard.data to mediumtext v1" 2024-09-11T07:58:26.126 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.88504349Z level=info msg="Executing migration" id="Add column updated_by in dashboard - v2" 2024-09-11T07:58:26.126 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.885886103Z level=info msg="Executing migration" id="Add column created_by in dashboard - v2" 2024-09-11T07:58:26.126 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.886734357Z level=info msg="Executing migration" id="Add column gnetId in dashboard" 2024-09-11T07:58:26.126 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.887589577Z level=info msg="Executing migration" id="Add index for gnetId in dashboard" 2024-09-11T07:58:26.126 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.888442959Z level=info msg="Executing migration" id="Add column plugin_id in dashboard" 2024-09-11T07:58:26.126 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.889291818Z level=info msg="Executing migration" id="Add index for plugin_id in dashboard" 2024-09-11T07:58:26.126 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.890133491Z level=info msg="Executing migration" id="Add index for dashboard_id in dashboard_tag" 2024-09-11T07:58:26.126 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.890982474Z level=info msg="Executing migration" id="Update dashboard table charset" 2024-09-11T07:58:26.126 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.891934081Z level=info msg="Executing migration" id="Update dashboard_tag table charset" 2024-09-11T07:58:26.126 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.8928275Z level=info msg="Executing migration" id="Add column folder_id in dashboard" 2024-09-11T07:58:26.127 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.893596785Z level=info msg="Executing migration" id="Add column isFolder in dashboard" 2024-09-11T07:58:26.127 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.896353134Z level=info msg="Executing migration" id="Add column has_acl in dashboard" 2024-09-11T07:58:26.127 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.899168977Z level=info msg="Executing migration" id="Add column uid in dashboard" 2024-09-11T07:58:26.127 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.899644278Z level=info msg="Executing migration" id="Update uid column values in dashboard" 2024-09-11T07:58:26.127 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.902384345Z level=info msg="Executing migration" id="Add unique index dashboard_org_id_uid" 2024-09-11T07:58:26.127 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.902785718Z level=info msg="Executing migration" id="Remove unique index org_id_slug" 2024-09-11T07:58:26.127 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.905130998Z level=info msg="Executing migration" id="Update dashboard title length" 2024-09-11T07:58:26.127 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.907904992Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_title_folder_id" 2024-09-11T07:58:26.127 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.908331062Z level=info msg="Executing migration" id="create dashboard_provisioning" 2024-09-11T07:58:26.127 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.911074227Z level=info msg="Executing migration" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2024-09-11T07:58:26.127 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.914015461Z level=info msg="Executing migration" id="create dashboard_provisioning v2" 2024-09-11T07:58:26.127 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.916712206Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2024-09-11T07:58:26.127 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.917115067Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2024-09-11T07:58:26.127 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.919886076Z level=info msg="Executing migration" id="copy dashboard_provisioning v1 to v2" 2024-09-11T07:58:26.128 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.92263063Z level=info msg="Executing migration" id="drop dashboard_provisioning_tmp_qwerty" 2024-09-11T07:58:26.128 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.925312701Z level=info msg="Executing migration" id="Add check_sum column" 2024-09-11T07:58:26.128 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.925811118Z level=info msg="Executing migration" id="Add index for dashboard_title" 2024-09-11T07:58:26.128 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.928590872Z level=info msg="Executing migration" id="delete tags for deleted dashboards" 2024-09-11T07:58:26.128 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.931323714Z level=info msg="Executing migration" id="delete stars for deleted dashboards" 2024-09-11T07:58:26.129 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.934099842Z level=info msg="Executing migration" id="Add index for dashboard_is_folder" 2024-09-11T07:58:26.129 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.934549011Z level=info msg="Executing migration" id="Add isPublic for dashboard" 2024-09-11T07:58:26.129 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.937424999Z level=info msg="Executing migration" id="create data_source table" 2024-09-11T07:58:26.129 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.938025627Z level=info msg="Executing migration" id="add index data_source.account_id" 2024-09-11T07:58:26.129 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.940064591Z level=info msg="Executing migration" id="add unique index data_source.account_id_name" 2024-09-11T07:58:26.129 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.942790052Z level=info msg="Executing migration" id="drop index IDX_data_source_account_id - v1" 2024-09-11T07:58:26.130 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.943329022Z level=info msg="Executing migration" id="drop index UQE_data_source_account_id_name - v1" 2024-09-11T07:58:26.130 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.946095605Z level=info msg="Executing migration" id="Rename table data_source to data_source_v1 - v1" 2024-09-11T07:58:26.130 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.948854119Z level=info msg="Executing migration" id="create data_source table v2" 2024-09-11T07:58:26.130 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.951686911Z level=info msg="Executing migration" id="create index IDX_data_source_org_id - v2" 2024-09-11T07:58:26.130 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.952122969Z level=info msg="Executing migration" id="create index UQE_data_source_org_id_name - v2" 2024-09-11T07:58:26.130 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.952549363Z level=info msg="Executing migration" id="Drop old table data_source_v1 #2" 2024-09-11T07:58:26.130 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.9529031Z level=info msg="Executing migration" id="Add column with_credentials" 2024-09-11T07:58:26.130 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.953515252Z level=info msg="Executing migration" id="Add secure json data column" 2024-09-11T07:58:26.130 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.954066194Z level=info msg="Executing migration" id="Update data_source table charset" 2024-09-11T07:58:26.130 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.954369138Z level=info msg="Executing migration" id="Update initial version to 1" 2024-09-11T07:58:26.130 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.954670118Z level=info msg="Executing migration" id="Add read_only data column" 2024-09-11T07:58:26.130 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.955221731Z level=info msg="Executing migration" id="Migrate logging ds to loki ds" 2024-09-11T07:58:26.130 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.955540484Z level=info msg="Executing migration" id="Update json_data with nulls" 2024-09-11T07:58:26.130 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.955844953Z level=info msg="Executing migration" id="Add uid column" 2024-09-11T07:58:26.130 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.956379059Z level=info msg="Executing migration" id="Update uid value" 2024-09-11T07:58:26.131 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.956691541Z level=info msg="Executing migration" id="Add unique index datasource_org_id_uid" 2024-09-11T07:58:26.131 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.95711223Z level=info msg="Executing migration" id="add unique index datasource_org_id_is_default" 2024-09-11T07:58:26.131 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.957537834Z level=info msg="Executing migration" id="create api_key table" 2024-09-11T07:58:26.131 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.957924525Z level=info msg="Executing migration" id="add index api_key.account_id" 2024-09-11T07:58:26.131 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.958355033Z level=info msg="Executing migration" id="add index api_key.key" 2024-09-11T07:58:26.131 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.958779237Z level=info msg="Executing migration" id="add index api_key.account_id_name" 2024-09-11T07:58:26.131 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.959249074Z level=info msg="Executing migration" id="drop index IDX_api_key_account_id - v1" 2024-09-11T07:58:26.131 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.959668938Z level=info msg="Executing migration" id="drop index UQE_api_key_key - v1" 2024-09-11T07:58:26.131 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.962456752Z level=info msg="Executing migration" id="drop index UQE_api_key_account_id_name - v1" 2024-09-11T07:58:26.131 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.96288509Z level=info msg="Executing migration" id="Rename table api_key to api_key_v1 - v1" 2024-09-11T07:58:26.131 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.965168036Z level=info msg="Executing migration" id="create api_key table v2" 2024-09-11T07:58:26.131 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.967841645Z level=info msg="Executing migration" id="create index IDX_api_key_org_id - v2" 2024-09-11T07:58:26.131 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.970472549Z level=info msg="Executing migration" id="create index UQE_api_key_key - v2" 2024-09-11T07:58:26.131 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.97318895Z level=info msg="Executing migration" id="create index UQE_api_key_org_id_name - v2" 2024-09-11T07:58:26.132 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.973736284Z level=info msg="Executing migration" id="copy api_key v1 to v2" 2024-09-11T07:58:26.132 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.974052766Z level=info msg="Executing migration" id="Drop old table api_key_v1" 2024-09-11T07:58:26.132 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.97624383Z level=info msg="Executing migration" id="Update api_key table charset" 2024-09-11T07:58:26.132 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.979061371Z level=info msg="Executing migration" id="Add expires to api_key table" 2024-09-11T07:58:26.132 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.981549356Z level=info msg="Executing migration" id="Add service account foreign key" 2024-09-11T07:58:26.132 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.982112378Z level=info msg="Executing migration" id="set service account foreign key to nil if 0" 2024-09-11T07:58:26.132 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.985040362Z level=info msg="Executing migration" id="Add last_used_at to api_key table" 2024-09-11T07:58:26.132 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.987558721Z level=info msg="Executing migration" id="Add is_revoked column to api_key table" 2024-09-11T07:58:26.132 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.990305837Z level=info msg="Executing migration" id="create dashboard_snapshot table v4" 2024-09-11T07:58:26.132 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.990714213Z level=info msg="Executing migration" id="drop table dashboard_snapshot_v4 #1" 2024-09-11T07:58:26.132 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.993344819Z level=info msg="Executing migration" id="create dashboard_snapshot table v5 #2" 2024-09-11T07:58:26.132 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.993755462Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_key - v5" 2024-09-11T07:58:26.132 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.994211369Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_delete_key - v5" 2024-09-11T07:58:26.133 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.995060996Z level=info msg="Executing migration" id="create index IDX_dashboard_snapshot_user_id - v5" 2024-09-11T07:58:26.133 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.995486307Z level=info msg="Executing migration" id="alter dashboard_snapshot to mediumtext v2" 2024-09-11T07:58:26.133 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.996359378Z level=info msg="Executing migration" id="Update dashboard_snapshot table charset" 2024-09-11T07:58:26.133 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.996861804Z level=info msg="Executing migration" id="Add column external_delete_url to dashboard_snapshots table" 2024-09-11T07:58:26.133 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.997436177Z level=info msg="Executing migration" id="Add encrypted dashboard json column" 2024-09-11T07:58:26.133 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.998214316Z level=info msg="Executing migration" id="Change dashboard_encrypted column to MEDIUMBLOB" 2024-09-11T07:58:26.133 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.998542716Z level=info msg="Executing migration" id="create quota table v1" 2024-09-11T07:58:26.133 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:25.999617162Z level=info msg="Executing migration" id="create index UQE_quota_org_id_user_id_target - v1" 2024-09-11T07:58:26.133 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.00024192Z level=info msg="Executing migration" id="Update quota table charset" 2024-09-11T07:58:26.133 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.001223099Z level=info msg="Executing migration" id="create plugin_setting table" 2024-09-11T07:58:26.134 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.001637675Z level=info msg="Executing migration" id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2024-09-11T07:58:26.134 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.002449996Z level=info msg="Executing migration" id="Add column plugin_version to plugin_settings" 2024-09-11T07:58:26.134 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.003326607Z level=info msg="Executing migration" id="Update plugin_setting table charset" 2024-09-11T07:58:26.134 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.004038413Z level=info msg="Executing migration" id="create session table" 2024-09-11T07:58:26.134 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.004462226Z level=info msg="Executing migration" id="Drop old table playlist table" 2024-09-11T07:58:26.134 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.005277359Z level=info msg="Executing migration" id="Drop old table playlist_item table" 2024-09-11T07:58:26.134 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.005593471Z level=info msg="Executing migration" id="create playlist table v2" 2024-09-11T07:58:26.134 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.006157009Z level=info msg="Executing migration" id="create playlist item table v2" 2024-09-11T07:58:26.134 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.007154279Z level=info msg="Executing migration" id="Update playlist table charset" 2024-09-11T07:58:26.134 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.007570487Z level=info msg="Executing migration" id="Update playlist_item table charset" 2024-09-11T07:58:26.134 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.008398556Z level=info msg="Executing migration" id="drop preferences table v2" 2024-09-11T07:58:26.134 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.008788341Z level=info msg="Executing migration" id="drop preferences table v3" 2024-09-11T07:58:26.134 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.009270602Z level=info msg="Executing migration" id="create preferences table v3" 2024-09-11T07:58:26.134 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.009714396Z level=info msg="Executing migration" id="Update preferences table charset" 2024-09-11T07:58:26.135 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.010125458Z level=info msg="Executing migration" id="Add column team_id in preferences" 2024-09-11T07:58:26.135 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.011441853Z level=info msg="Executing migration" id="Update team_id column values in preferences" 2024-09-11T07:58:26.135 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.011875459Z level=info msg="Executing migration" id="Add column week_start in preferences" 2024-09-11T07:58:26.135 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.012733896Z level=info msg="Executing migration" id="Add column preferences.json_data" 2024-09-11T07:58:26.135 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.013560542Z level=info msg="Executing migration" id="alter preferences.json_data to mediumtext v1" 2024-09-11T07:58:26.135 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.014413914Z level=info msg="Executing migration" id="Add preferences index org_id" 2024-09-11T07:58:26.135 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.015275944Z level=info msg="Executing migration" id="Add preferences index user_id" 2024-09-11T07:58:26.135 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.016172286Z level=info msg="Executing migration" id="create alert table v1" 2024-09-11T07:58:26.135 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.016944986Z level=info msg="Executing migration" id="add index alert org_id & id " 2024-09-11T07:58:26.135 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.017474125Z level=info msg="Executing migration" id="add index alert state" 2024-09-11T07:58:26.135 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.020182907Z level=info msg="Executing migration" id="add index alert dashboard_id" 2024-09-11T07:58:26.135 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.023087917Z level=info msg="Executing migration" id="Create alert_rule_tag table v1" 2024-09-11T07:58:26.135 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.025664931Z level=info msg="Executing migration" id="Add unique index alert_rule_tag.alert_id_tag_id" 2024-09-11T07:58:26.135 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.028519279Z level=info msg="Executing migration" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2024-09-11T07:58:26.136 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.031372047Z level=info msg="Executing migration" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2024-09-11T07:58:26.136 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.034189089Z level=info msg="Executing migration" id="Create alert_rule_tag table v2" 2024-09-11T07:58:26.136 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.0370258Z level=info msg="Executing migration" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2024-09-11T07:58:26.136 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.0397753Z level=info msg="Executing migration" id="copy alert_rule_tag v1 to v2" 2024-09-11T07:58:26.136 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.043115228Z level=info msg="Executing migration" id="drop table alert_rule_tag_v1" 2024-09-11T07:58:26.136 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.04502488Z level=info msg="Executing migration" id="create alert_notification table v1" 2024-09-11T07:58:26.136 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.047726856Z level=info msg="Executing migration" id="Add column is_default" 2024-09-11T07:58:26.136 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.050570745Z level=info msg="Executing migration" id="Add column frequency" 2024-09-11T07:58:26.136 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.05335643Z level=info msg="Executing migration" id="Add column send_reminder" 2024-09-11T07:58:26.136 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.055785219Z level=info msg="Executing migration" id="Add column disable_resolve_message" 2024-09-11T07:58:26.136 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.058677226Z level=info msg="Executing migration" id="add index alert_notification org_id & name" 2024-09-11T07:58:26.136 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.061359151Z level=info msg="Executing migration" id="Update alert table charset" 2024-09-11T07:58:26.136 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.064178494Z level=info msg="Executing migration" id="Update alert_notification table charset" 2024-09-11T07:58:26.136 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.066850413Z level=info msg="Executing migration" id="create notification_journal table v1" 2024-09-11T07:58:26.137 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.069346438Z level=info msg="Executing migration" id="add index notification_journal org_id & alert_id & notifier_id" 2024-09-11T07:58:26.137 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.07222708Z level=info msg="Executing migration" id="drop alert_notification_journal" 2024-09-11T07:58:26.137 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.074831582Z level=info msg="Executing migration" id="create alert_notification_state table v1" 2024-09-11T07:58:26.137 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.07748516Z level=info msg="Executing migration" id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-09-11T07:58:26.137 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.080281723Z level=info msg="Executing migration" id="Add for to alert table" 2024-09-11T07:58:26.137 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.082935517Z level=info msg="Executing migration" id="Add column uid in alert_notification" 2024-09-11T07:58:26.137 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.085739719Z level=info msg="Executing migration" id="Update uid column values in alert_notification" 2024-09-11T07:58:26.137 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.088350182Z level=info msg="Executing migration" id="Add unique index alert_notification_org_id_uid" 2024-09-11T07:58:26.137 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.088881381Z level=info msg="Executing migration" id="Remove unique index org_id_name" 2024-09-11T07:58:26.137 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.091620172Z level=info msg="Executing migration" id="Add column secure_settings in alert_notification" 2024-09-11T07:58:26.137 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.094285321Z level=info msg="Executing migration" id="alter alert.settings to mediumtext" 2024-09-11T07:58:26.137 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.096856922Z level=info msg="Executing migration" id="Add non-unique index alert_notification_state_alert_id" 2024-09-11T07:58:26.137 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.097457914Z level=info msg="Executing migration" id="Add non-unique index alert_rule_tag_alert_id" 2024-09-11T07:58:26.138 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.10023805Z level=info msg="Executing migration" id="Drop old annotation table v4" 2024-09-11T07:58:26.138 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.102726808Z level=info msg="Executing migration" id="create annotation table v5" 2024-09-11T07:58:26.138 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.103243905Z level=info msg="Executing migration" id="add index annotation 0 v3" 2024-09-11T07:58:26.138 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.106083754Z level=info msg="Executing migration" id="add index annotation 1 v3" 2024-09-11T07:58:26.138 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.108858328Z level=info msg="Executing migration" id="add index annotation 2 v3" 2024-09-11T07:58:26.138 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.111566176Z level=info msg="Executing migration" id="add index annotation 3 v3" 2024-09-11T07:58:26.138 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.114126413Z level=info msg="Executing migration" id="add index annotation 4 v3" 2024-09-11T07:58:26.138 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.114910259Z level=info msg="Executing migration" id="Update annotation table charset" 2024-09-11T07:58:26.138 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.115248664Z level=info msg="Executing migration" id="Add column region_id to annotation table" 2024-09-11T07:58:26.138 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.115951625Z level=info msg="Executing migration" id="Drop category_id index" 2024-09-11T07:58:26.138 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.116387696Z level=info msg="Executing migration" id="Add column tags to annotation table" 2024-09-11T07:58:26.138 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.117098163Z level=info msg="Executing migration" id="Create annotation_tag table v2" 2024-09-11T07:58:26.138 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.117561733Z level=info msg="Executing migration" id="Add unique index annotation_tag.annotation_id_tag_id" 2024-09-11T07:58:26.138 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.118027394Z level=info msg="Executing migration" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-09-11T07:58:26.139 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.118453023Z level=info msg="Executing migration" id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-09-11T07:58:26.139 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.120031552Z level=info msg="Executing migration" id="Create annotation_tag table v3" 2024-09-11T07:58:26.139 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.120427525Z level=info msg="Executing migration" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-09-11T07:58:26.139 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.120867049Z level=info msg="Executing migration" id="copy annotation_tag v2 to v3" 2024-09-11T07:58:26.139 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.121206568Z level=info msg="Executing migration" id="drop table annotation_tag_v2" 2024-09-11T07:58:26.139 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.121573435Z level=info msg="Executing migration" id="Update alert annotations and set TEXT to empty" 2024-09-11T07:58:26.139 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.121855094Z level=info msg="Executing migration" id="Add created time to annotation table" 2024-09-11T07:58:26.139 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.122558898Z level=info msg="Executing migration" id="Add updated time to annotation table" 2024-09-11T07:58:26.139 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.12326239Z level=info msg="Executing migration" id="Add index for created in annotation table" 2024-09-11T07:58:26.139 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.123698186Z level=info msg="Executing migration" id="Add index for updated in annotation table" 2024-09-11T07:58:26.139 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.12411314Z level=info msg="Executing migration" id="Convert existing annotations from seconds to milliseconds" 2024-09-11T07:58:26.139 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.124440956Z level=info msg="Executing migration" id="Add epoch_end column" 2024-09-11T07:58:26.139 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.125429715Z level=info msg="Executing migration" id="Add index for epoch_end" 2024-09-11T07:58:26.139 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.125997952Z level=info msg="Executing migration" id="Make epoch_end the same as epoch" 2024-09-11T07:58:26.140 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.126321679Z level=info msg="Executing migration" id="Move region to single row" 2024-09-11T07:58:26.140 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.126646508Z level=info msg="Executing migration" id="Remove index org_id_epoch from annotation table" 2024-09-11T07:58:26.140 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.127017172Z level=info msg="Executing migration" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-09-11T07:58:26.140 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.127417854Z level=info msg="Executing migration" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-09-11T07:58:26.140 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.127797617Z level=info msg="Executing migration" id="Add index for org_id_epoch_end_epoch on annotation table" 2024-09-11T07:58:26.140 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.128207688Z level=info msg="Executing migration" id="Remove index org_id_epoch_epoch_end from annotation table" 2024-09-11T07:58:26.140 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.128612005Z level=info msg="Executing migration" id="Add index for alert_id on annotation table" 2024-09-11T07:58:26.375 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-mon[27311]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:58:26.375 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:26.375 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:26.375 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:26.375 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:26.376 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.12898244Z level=info msg="Executing migration" id="Increase tags column to length 4096" 2024-09-11T07:58:26.376 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.129301068Z level=info msg="Executing migration" id="create test_data table" 2024-09-11T07:58:26.376 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.129690553Z level=info msg="Executing migration" id="create dashboard_version table v1" 2024-09-11T07:58:26.376 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.130068531Z level=info msg="Executing migration" id="add index dashboard_version.dashboard_id" 2024-09-11T07:58:26.376 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.130477597Z level=info msg="Executing migration" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-09-11T07:58:26.376 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.130878016Z level=info msg="Executing migration" id="Set dashboard version to 1 where 0" 2024-09-11T07:58:26.376 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.131181465Z level=info msg="Executing migration" id="save existing dashboard data in dashboard_version table v1" 2024-09-11T07:58:26.376 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.1340894Z level=info msg="Executing migration" id="alter dashboard_version.data to mediumtext v1" 2024-09-11T07:58:26.376 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.136774481Z level=info msg="Executing migration" id="create team table" 2024-09-11T07:58:26.377 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.137614872Z level=info msg="Executing migration" id="add index team.org_id" 2024-09-11T07:58:26.377 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.138458355Z level=info msg="Executing migration" id="add unique index team_org_id_name" 2024-09-11T07:58:26.377 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.1394097Z level=info msg="Executing migration" id="create team member table" 2024-09-11T07:58:26.377 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.140079012Z level=info msg="Executing migration" id="add index team_member.org_id" 2024-09-11T07:58:26.377 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.141028096Z level=info msg="Executing migration" id="add unique index team_member_org_id_team_id_user_id" 2024-09-11T07:58:26.377 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.141686531Z level=info msg="Executing migration" id="add index team_member.team_id" 2024-09-11T07:58:26.377 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.142508089Z level=info msg="Executing migration" id="Add column email to team table" 2024-09-11T07:58:26.378 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.143608533Z level=info msg="Executing migration" id="Add column external to team_member table" 2024-09-11T07:58:26.378 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.144957524Z level=info msg="Executing migration" id="Add column permission to team_member table" 2024-09-11T07:58:26.378 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.146052553Z level=info msg="Executing migration" id="create dashboard acl table" 2024-09-11T07:58:26.378 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.14689284Z level=info msg="Executing migration" id="add index dashboard_acl_dashboard_id" 2024-09-11T07:58:26.378 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.147742542Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_user_id" 2024-09-11T07:58:26.378 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.148483826Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_team_id" 2024-09-11T07:58:26.378 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.148911418Z level=info msg="Executing migration" id="add index dashboard_acl_user_id" 2024-09-11T07:58:26.378 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.149730546Z level=info msg="Executing migration" id="add index dashboard_acl_team_id" 2024-09-11T07:58:26.378 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.150539935Z level=info msg="Executing migration" id="add index dashboard_acl_org_id_role" 2024-09-11T07:58:26.378 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.151443133Z level=info msg="Executing migration" id="add index dashboard_permission" 2024-09-11T07:58:26.378 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.152000002Z level=info msg="Executing migration" id="save default acl rules in dashboard_acl table" 2024-09-11T07:58:26.379 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.152790974Z level=info msg="Executing migration" id="delete acl rules for deleted dashboards and folders" 2024-09-11T07:58:26.379 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.153594017Z level=info msg="Executing migration" id="create tag table" 2024-09-11T07:58:26.379 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.154058019Z level=info msg="Executing migration" id="add index tag.key_value" 2024-09-11T07:58:26.379 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.154504358Z level=info msg="Executing migration" id="create login attempt table" 2024-09-11T07:58:26.379 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.15490633Z level=info msg="Executing migration" id="add index login_attempt.username" 2024-09-11T07:58:26.379 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.155744709Z level=info msg="Executing migration" id="drop index IDX_login_attempt_username - v1" 2024-09-11T07:58:26.379 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.15663402Z level=info msg="Executing migration" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-09-11T07:58:26.379 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.15859825Z level=info msg="Executing migration" id="create login_attempt v2" 2024-09-11T07:58:26.379 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.15898767Z level=info msg="Executing migration" id="create index IDX_login_attempt_username - v2" 2024-09-11T07:58:26.379 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.160035592Z level=info msg="Executing migration" id="copy login_attempt v1 to v2" 2024-09-11T07:58:26.379 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.160705266Z level=info msg="Executing migration" id="drop login_attempt_tmp_qwerty" 2024-09-11T07:58:26.379 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.161536337Z level=info msg="Executing migration" id="create user auth table" 2024-09-11T07:58:26.380 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.161962812Z level=info msg="Executing migration" id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-09-11T07:58:26.380 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.162412739Z level=info msg="Executing migration" id="alter user_auth.auth_id to length 190" 2024-09-11T07:58:26.380 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.163220626Z level=info msg="Executing migration" id="Add OAuth access token to user_auth" 2024-09-11T07:58:26.380 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.164029555Z level=info msg="Executing migration" id="Add OAuth refresh token to user_auth" 2024-09-11T07:58:26.380 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.164834918Z level=info msg="Executing migration" id="Add OAuth token type to user_auth" 2024-09-11T07:58:26.380 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.167620802Z level=info msg="Executing migration" id="Add OAuth expiry to user_auth" 2024-09-11T07:58:26.380 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.168429381Z level=info msg="Executing migration" id="Add index to user_id column in user_auth" 2024-09-11T07:58:26.380 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.171077706Z level=info msg="Executing migration" id="Add OAuth ID token to user_auth" 2024-09-11T07:58:26.380 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.173917084Z level=info msg="Executing migration" id="create server_lock table" 2024-09-11T07:58:26.380 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.176589321Z level=info msg="Executing migration" id="add index server_lock.operation_uid" 2024-09-11T07:58:26.380 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.179276028Z level=info msg="Executing migration" id="create user auth token table" 2024-09-11T07:58:26.380 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.181980203Z level=info msg="Executing migration" id="add unique index user_auth_token.auth_token" 2024-09-11T07:58:26.380 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.184744414Z level=info msg="Executing migration" id="add unique index user_auth_token.prev_auth_token" 2024-09-11T07:58:26.381 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.187385219Z level=info msg="Executing migration" id="add index user_auth_token.user_id" 2024-09-11T07:58:26.381 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.187973073Z level=info msg="Executing migration" id="Add revoked_at to the user auth token" 2024-09-11T07:58:26.381 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.190700757Z level=info msg="Executing migration" id="create cache_data table" 2024-09-11T07:58:26.381 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.193619874Z level=info msg="Executing migration" id="add unique index cache_data.cache_key" 2024-09-11T07:58:26.381 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.196222863Z level=info msg="Executing migration" id="create short_url table v1" 2024-09-11T07:58:26.382 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.199037638Z level=info msg="Executing migration" id="add index short_url.org_id-uid" 2024-09-11T07:58:26.382 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.20174777Z level=info msg="Executing migration" id="alter table short_url alter column created_by type to bigint" 2024-09-11T07:58:26.382 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.204403756Z level=info msg="Executing migration" id="delete alert_definition table" 2024-09-11T07:58:26.382 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.207387524Z level=info msg="Executing migration" id="recreate alert_definition table" 2024-09-11T07:58:26.382 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.210366031Z level=info msg="Executing migration" id="add index in alert_definition on org_id and title columns" 2024-09-11T07:58:26.382 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.213049766Z level=info msg="Executing migration" id="add index in alert_definition on org_id and uid columns" 2024-09-11T07:58:26.382 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.215594849Z level=info msg="Executing migration" id="alter alert_definition table data column to mediumtext in mysql" 2024-09-11T07:58:26.382 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.218350011Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and title columns" 2024-09-11T07:58:26.382 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.221174071Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and uid columns" 2024-09-11T07:58:26.383 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.223788121Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and title columns" 2024-09-11T07:58:26.383 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.226547775Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and uid columns" 2024-09-11T07:58:26.383 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.229162488Z level=info msg="Executing migration" id="Add column paused in alert_definition" 2024-09-11T07:58:26.383 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.231930208Z level=info msg="Executing migration" id="drop alert_definition table" 2024-09-11T07:58:26.383 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.234522893Z level=info msg="Executing migration" id="delete alert_definition_version table" 2024-09-11T07:58:26.383 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.237249087Z level=info msg="Executing migration" id="recreate alert_definition_version table" 2024-09-11T07:58:26.383 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.240074942Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-09-11T07:58:26.383 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.242688646Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-09-11T07:58:26.383 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.245384244Z level=info msg="Executing migration" id="alter alert_definition_version table data column to mediumtext in mysql" 2024-09-11T07:58:26.383 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.247940866Z level=info msg="Executing migration" id="drop alert_definition_version table" 2024-09-11T07:58:26.383 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.250586405Z level=info msg="Executing migration" id="create alert_instance table" 2024-09-11T07:58:26.383 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.253399698Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-09-11T07:58:26.383 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.25625102Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, current_state columns" 2024-09-11T07:58:26.383 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.256863159Z level=info msg="Executing migration" id="add column current_state_end to alert_instance" 2024-09-11T07:58:26.384 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.257716946Z level=info msg="Executing migration" id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-09-11T07:58:26.384 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.258658114Z level=info msg="Executing migration" id="remove index def_org_id, current_state on alert_instance" 2024-09-11T07:58:26.384 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.259364836Z level=info msg="Executing migration" id="rename def_org_id to rule_org_id in alert_instance" 2024-09-11T07:58:26.384 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.262816268Z level=info msg="Executing migration" id="rename def_uid to rule_uid in alert_instance" 2024-09-11T07:58:26.384 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.266124912Z level=info msg="Executing migration" id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-09-11T07:58:26.384 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.266966293Z level=info msg="Executing migration" id="add index rule_org_id, current_state on alert_instance" 2024-09-11T07:58:26.384 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.267796272Z level=info msg="Executing migration" id="add current_reason column related to current_state" 2024-09-11T07:58:26.384 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.268640708Z level=info msg="Executing migration" id="create alert_rule table" 2024-09-11T07:58:26.384 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.269472081Z level=info msg="Executing migration" id="add index in alert_rule on org_id and title columns" 2024-09-11T07:58:26.384 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.270334803Z level=info msg="Executing migration" id="add index in alert_rule on org_id and uid columns" 2024-09-11T07:58:26.384 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.271155595Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-09-11T07:58:26.384 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.271998067Z level=info msg="Executing migration" id="alter alert_rule table data column to mediumtext in mysql" 2024-09-11T07:58:26.385 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.272917934Z level=info msg="Executing migration" id="add column for to alert_rule" 2024-09-11T07:58:26.385 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.27391123Z level=info msg="Executing migration" id="add column annotations to alert_rule" 2024-09-11T07:58:26.385 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.274755232Z level=info msg="Executing migration" id="add column labels to alert_rule" 2024-09-11T07:58:26.385 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.275616785Z level=info msg="Executing migration" id="remove unique index from alert_rule on org_id, title columns" 2024-09-11T07:58:26.385 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.276491961Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-09-11T07:58:26.385 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.277421329Z level=info msg="Executing migration" id="add dashboard_uid column to alert_rule" 2024-09-11T07:58:26.385 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.278420572Z level=info msg="Executing migration" id="add panel_id column to alert_rule" 2024-09-11T07:58:26.385 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.279290504Z level=info msg="Executing migration" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-09-11T07:58:26.385 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.28013992Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule" 2024-09-11T07:58:26.385 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.280993429Z level=info msg="Executing migration" id="add is_paused column to alert_rule table" 2024-09-11T07:58:26.385 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.28188032Z level=info msg="Executing migration" id="fix is_paused column for alert_rule table" 2024-09-11T07:58:26.385 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.283038477Z level=info msg="Executing migration" id="create alert_rule_version table" 2024-09-11T07:58:26.385 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.283603343Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-09-11T07:58:26.385 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.284457306Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-09-11T07:58:26.386 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.285284787Z level=info msg="Executing migration" id="alter alert_rule_version table data column to mediumtext in mysql" 2024-09-11T07:58:26.386 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.286126614Z level=info msg="Executing migration" id="add column for to alert_rule_version" 2024-09-11T07:58:26.386 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.287029309Z level=info msg="Executing migration" id="add column annotations to alert_rule_version" 2024-09-11T07:58:26.386 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.287915015Z level=info msg="Executing migration" id="add column labels to alert_rule_version" 2024-09-11T07:58:26.386 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.288795167Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule_version" 2024-09-11T07:58:26.386 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.289672338Z level=info msg="Executing migration" id="add is_paused column to alert_rule_versions table" 2024-09-11T07:58:26.386 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.290566147Z level=info msg="Executing migration" id="fix is_paused column for alert_rule_version table" 2024-09-11T07:58:26.386 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.291434787Z level=info msg="Executing migration" id=create_alert_configuration_table 2024-09-11T07:58:26.386 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.294152891Z level=info msg="Executing migration" id="Add column default in alert_configuration" 2024-09-11T07:58:26.386 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.296927095Z level=info msg="Executing migration" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-09-11T07:58:26.386 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.299687736Z level=info msg="Executing migration" id="add column org_id in alert_configuration" 2024-09-11T07:58:26.386 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.302439179Z level=info msg="Executing migration" id="add index in alert_configuration table on org_id column" 2024-09-11T07:58:26.386 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.305115493Z level=info msg="Executing migration" id="add configuration_hash column to alert_configuration" 2024-09-11T07:58:26.386 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.307906876Z level=info msg="Executing migration" id=create_ngalert_configuration_table 2024-09-11T07:58:26.387 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.310778267Z level=info msg="Executing migration" id="add index in ngalert_configuration on org_id column" 2024-09-11T07:58:26.387 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.313528117Z level=info msg="Executing migration" id="add column send_alerts_to in ngalert_configuration" 2024-09-11T07:58:26.387 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.316319728Z level=info msg="Executing migration" id="create provenance_type table" 2024-09-11T07:58:26.387 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.319254555Z level=info msg="Executing migration" id="add index to uniquify (record_key, record_type, org_id) columns" 2024-09-11T07:58:26.387 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.321983474Z level=info msg="Executing migration" id="create alert_image table" 2024-09-11T07:58:26.387 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.324813877Z level=info msg="Executing migration" id="add unique index on token to alert_image table" 2024-09-11T07:58:26.387 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.327542679Z level=info msg="Executing migration" id="support longer URLs in alert_image table" 2024-09-11T07:58:26.387 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.330259742Z level=info msg="Executing migration" id=create_alert_configuration_history_table 2024-09-11T07:58:26.387 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.333032445Z level=info msg="Executing migration" id="drop non-unique orgID index on alert_configuration" 2024-09-11T07:58:26.387 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.33570561Z level=info msg="Executing migration" id="drop unique orgID index on alert_configuration if exists" 2024-09-11T07:58:26.387 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.335786598Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop unique orgID index on alert_configuration if exists" 2024-09-11T07:58:26.387 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.338487857Z level=info msg="Executing migration" id="extract alertmanager configuration history to separate table" 2024-09-11T07:58:26.387 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.341212038Z level=info msg="Executing migration" id="add unique index on orgID to alert_configuration" 2024-09-11T07:58:26.388 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.343960611Z level=info msg="Executing migration" id="clear migration entry \"remove unified alerting data\"" 2024-09-11T07:58:26.388 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.344094422Z level=info msg="Executing migration" id="move dashboard alerts to unified alerting" 2024-09-11T07:58:26.388 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.344223589Z level=info msg="alerts found to migrate" alerts=0 2024-09-11T07:58:26.388 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.347183694Z level=info msg="Executing migration" id="create library_element table v1" 2024-09-11T07:58:26.388 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.349247339Z level=info msg="Executing migration" id="add index library_element org_id-folder_id-name-kind" 2024-09-11T07:58:26.388 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.351899159Z level=info msg="Executing migration" id="create library_element_connection table v1" 2024-09-11T07:58:26.388 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.354464561Z level=info msg="Executing migration" id="add index library_element_connection element_id-kind-connection_id" 2024-09-11T07:58:26.388 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.355046046Z level=info msg="Executing migration" id="add unique index library_element org_id_uid" 2024-09-11T07:58:26.388 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.355646939Z level=info msg="Executing migration" id="increase max description length to 2048" 2024-09-11T07:58:26.388 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.355991572Z level=info msg="Executing migration" id="clone move dashboard alerts to unified alerting" 2024-09-11T07:58:26.388 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.356643349Z level=info msg="Executing migration" id="create data_keys table" 2024-09-11T07:58:26.388 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.357210492Z level=info msg="Executing migration" id="create secrets table" 2024-09-11T07:58:26.388 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.35774562Z level=info msg="Executing migration" id="rename data_keys name column to id" 2024-09-11T07:58:26.389 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.36224595Z level=info msg="Executing migration" id="add name column into data_keys" 2024-09-11T07:58:26.389 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.36312584Z level=info msg="Executing migration" id="copy data_keys id column values into name" 2024-09-11T07:58:26.389 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.363432001Z level=info msg="Executing migration" id="rename data_keys name column to label" 2024-09-11T07:58:26.389 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.367675897Z level=info msg="Executing migration" id="rename data_keys id column back to name" 2024-09-11T07:58:26.389 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.371914755Z level=info msg="Executing migration" id="create kv_store table v1" 2024-09-11T07:58:26.389 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.372339057Z level=info msg="Executing migration" id="add index kv_store.org_id-namespace-key" 2024-09-11T07:58:26.389 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.372760516Z level=info msg="Executing migration" id="update dashboard_uid and panel_id from existing annotations" 2024-09-11T07:58:26.389 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.373073164Z level=info msg="Executing migration" id="create permission table" 2024-09-11T07:58:26.389 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.373453724Z level=info msg="Executing migration" id="add unique index permission.role_id" 2024-09-11T07:58:26.389 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.373855614Z level=info msg="Executing migration" id="add unique index role_id_action_scope" 2024-09-11T07:58:26.389 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.374291167Z level=info msg="Executing migration" id="create role table" 2024-09-11T07:58:26.389 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.374667482Z level=info msg="Executing migration" id="add column display_name" 2024-09-11T07:58:26.389 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.375691438Z level=info msg="Executing migration" id="add column group_name" 2024-09-11T07:58:26.390 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.376640635Z level=info msg="Executing migration" id="add index role.org_id" 2024-09-11T07:58:26.390 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.377073959Z level=info msg="Executing migration" id="add unique index role_org_id_name" 2024-09-11T07:58:26.390 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.37749555Z level=info msg="Executing migration" id="add index role_org_id_uid" 2024-09-11T07:58:26.390 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.377904513Z level=info msg="Executing migration" id="create team role table" 2024-09-11T07:58:26.390 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.378287074Z level=info msg="Executing migration" id="add index team_role.org_id" 2024-09-11T07:58:26.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:26 smithi110 ceph-mon[25392]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:58:26.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:26 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:26.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:26 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:26.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:26 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:26.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:26 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:26.468 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:26 smithi110 ceph-mon[31709]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:58:26.469 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:26 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:26.469 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:26 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:26.469 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:26 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:26.469 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:26 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:26.728 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.378775496Z level=info msg="Executing migration" id="add unique index team_role_org_id_team_id_role_id" 2024-09-11T07:58:26.728 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.379301857Z level=info msg="Executing migration" id="add index team_role.team_id" 2024-09-11T07:58:26.729 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.379778906Z level=info msg="Executing migration" id="create user role table" 2024-09-11T07:58:26.729 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.380202465Z level=info msg="Executing migration" id="add index user_role.org_id" 2024-09-11T07:58:26.729 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.38062707Z level=info msg="Executing migration" id="add unique index user_role_org_id_user_id_role_id" 2024-09-11T07:58:26.729 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.381057686Z level=info msg="Executing migration" id="add index user_role.user_id" 2024-09-11T07:58:26.729 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.381484911Z level=info msg="Executing migration" id="create builtin role table" 2024-09-11T07:58:26.729 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.381886263Z level=info msg="Executing migration" id="add index builtin_role.role_id" 2024-09-11T07:58:26.729 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.382701832Z level=info msg="Executing migration" id="add index builtin_role.name" 2024-09-11T07:58:26.729 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.383119969Z level=info msg="Executing migration" id="Add column org_id to builtin_role table" 2024-09-11T07:58:26.729 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.384100068Z level=info msg="Executing migration" id="add index builtin_role.org_id" 2024-09-11T07:58:26.729 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.384559378Z level=info msg="Executing migration" id="add unique index builtin_role_org_id_role_id_role" 2024-09-11T07:58:26.729 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.385503069Z level=info msg="Executing migration" id="Remove unique index role_org_id_uid" 2024-09-11T07:58:26.730 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.386341109Z level=info msg="Executing migration" id="add unique index role.uid" 2024-09-11T07:58:26.730 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.387220746Z level=info msg="Executing migration" id="create seed assignment table" 2024-09-11T07:58:26.730 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.387626605Z level=info msg="Executing migration" id="add unique index builtin_role_role_name" 2024-09-11T07:58:26.730 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.388473394Z level=info msg="Executing migration" id="add column hidden to role table" 2024-09-11T07:58:26.730 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.389468937Z level=info msg="Executing migration" id="create query_history table v1" 2024-09-11T07:58:26.730 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.392385295Z level=info msg="Executing migration" id="add index query_history.org_id-created_by-datasource_uid" 2024-09-11T07:58:26.731 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.393079953Z level=info msg="Executing migration" id="alter table query_history alter column created_by type to bigint" 2024-09-11T07:58:26.731 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.395834364Z level=info msg="Executing migration" id="teams permissions migration" 2024-09-11T07:58:26.731 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.398763038Z level=info msg="Executing migration" id="dashboard permissions" 2024-09-11T07:58:26.731 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.401569142Z level=info msg="Executing migration" id="dashboard permissions uid scopes" 2024-09-11T07:58:26.731 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.404295475Z level=info msg="Executing migration" id="drop managed folder create actions" 2024-09-11T07:58:26.731 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.407162127Z level=info msg="Executing migration" id="alerting notification permissions" 2024-09-11T07:58:26.731 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.409914107Z level=info msg="Executing migration" id="create query_history_star table v1" 2024-09-11T07:58:26.732 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.412727991Z level=info msg="Executing migration" id="add index query_history.user_id-query_uid" 2024-09-11T07:58:26.732 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.41323863Z level=info msg="Executing migration" id="add column org_id in query_history_star" 2024-09-11T07:58:26.732 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.416071794Z level=info msg="Executing migration" id="alter table query_history_star_mig column user_id type to bigint" 2024-09-11T07:58:26.732 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.4188716Z level=info msg="Executing migration" id="create correlation table v1" 2024-09-11T07:58:26.732 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.421750842Z level=info msg="Executing migration" id="add index correlations.uid" 2024-09-11T07:58:26.732 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.422406189Z level=info msg="Executing migration" id="add index correlations.source_uid" 2024-09-11T07:58:26.732 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.425362853Z level=info msg="Executing migration" id="add correlation config column" 2024-09-11T07:58:26.732 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.428074259Z level=info msg="Executing migration" id="create entity_events table" 2024-09-11T07:58:26.732 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.430936286Z level=info msg="Executing migration" id="create dashboard public config v1" 2024-09-11T07:58:26.732 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.431387547Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-11T07:58:26.732 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.431471288Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-11T07:58:26.732 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.434183067Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-11T07:58:26.732 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.434289004Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-11T07:58:26.732 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.434585341Z level=info msg="Executing migration" id="Drop old dashboard public config table" 2024-09-11T07:58:26.733 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.435128466Z level=info msg="Executing migration" id="recreate dashboard public config v1" 2024-09-11T07:58:26.733 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.4355693Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v1" 2024-09-11T07:58:26.733 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.436017983Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-11T07:58:26.733 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.438811235Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v2" 2024-09-11T07:58:26.733 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.441484168Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-11T07:58:26.733 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.443988334Z level=info msg="Executing migration" id="Drop public config table" 2024-09-11T07:58:26.733 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.444462177Z level=info msg="Executing migration" id="Recreate dashboard public config v2" 2024-09-11T07:58:26.733 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.447131413Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v2" 2024-09-11T07:58:26.733 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.44777984Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-11T07:58:26.733 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.449732712Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_access_token - v2" 2024-09-11T07:58:26.734 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.452355563Z level=info msg="Executing migration" id="Rename table dashboard_public_config to dashboard_public - v2" 2024-09-11T07:58:26.734 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.455635147Z level=info msg="Executing migration" id="add annotations_enabled column" 2024-09-11T07:58:26.734 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.458245464Z level=info msg="Executing migration" id="add time_selection_enabled column" 2024-09-11T07:58:26.734 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.460908563Z level=info msg="Executing migration" id="delete orphaned public dashboards" 2024-09-11T07:58:26.734 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.461277526Z level=info msg="Executing migration" id="add share column" 2024-09-11T07:58:26.734 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.463507703Z level=info msg="Executing migration" id="create default alerting folders" 2024-09-11T07:58:26.734 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.466192756Z level=info msg="Executing migration" id="create file table" 2024-09-11T07:58:26.734 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.466839589Z level=info msg="Executing migration" id="file table idx: path natural pk" 2024-09-11T07:58:26.734 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.469585743Z level=info msg="Executing migration" id="file table idx: parent_folder_path_hash fast folder retrieval" 2024-09-11T07:58:26.734 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.470298118Z level=info msg="Executing migration" id="create file_meta table" 2024-09-11T07:58:26.734 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.471107316Z level=info msg="Executing migration" id="file table idx: path key" 2024-09-11T07:58:26.734 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.471706235Z level=info msg="Executing migration" id="set path collation in file table" 2024-09-11T07:58:26.734 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.472084128Z level=info msg="Executing migration" id="managed permissions migration" 2024-09-11T07:58:26.735 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.472614111Z level=info msg="Executing migration" id="managed folder permissions alert actions migration" 2024-09-11T07:58:26.735 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.473000105Z level=info msg="Executing migration" id="RBAC action name migrator" 2024-09-11T07:58:26.735 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.473602346Z level=info msg="Executing migration" id="Add UID column to playlist" 2024-09-11T07:58:26.735 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.475215119Z level=info msg="Executing migration" id="Update uid column values in playlist" 2024-09-11T07:58:26.735 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.475626214Z level=info msg="Executing migration" id="Add index for uid in playlist" 2024-09-11T07:58:26.735 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.476259071Z level=info msg="Executing migration" id="update group index for alert rules" 2024-09-11T07:58:26.735 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.476749361Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated migration" 2024-09-11T07:58:26.735 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.477088674Z level=info msg="Executing migration" id="admin only folder/dashboard permission" 2024-09-11T07:58:26.735 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.477550963Z level=info msg="Executing migration" id="add action column to seed_assignment" 2024-09-11T07:58:26.735 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.478662308Z level=info msg="Executing migration" id="add scope column to seed_assignment" 2024-09-11T07:58:26.735 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.479842699Z level=info msg="Executing migration" id="remove unique index builtin_role_role_name before nullable update" 2024-09-11T07:58:26.735 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.480387284Z level=info msg="Executing migration" id="update seed_assignment role_name column to nullable" 2024-09-11T07:58:26.735 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.490313611Z level=info msg="Executing migration" id="add unique index builtin_role_name back" 2024-09-11T07:58:26.735 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.49109342Z level=info msg="Executing migration" id="add unique index builtin_role_action_scope" 2024-09-11T07:58:26.736 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.491614424Z level=info msg="Executing migration" id="add primary key to seed_assigment" 2024-09-11T07:58:26.736 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.495229671Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated fixed migration" 2024-09-11T07:58:26.736 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.496042926Z level=info msg="Executing migration" id="migrate external alertmanagers to datsourcse" 2024-09-11T07:58:26.736 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.496464431Z level=info msg="Executing migration" id="create folder table" 2024-09-11T07:58:26.736 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.496883792Z level=info msg="Executing migration" id="Add index for parent_uid" 2024-09-11T07:58:26.736 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.497382672Z level=info msg="Executing migration" id="Add unique index for folder.uid and folder.org_id" 2024-09-11T07:58:26.736 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.497885177Z level=info msg="Executing migration" id="Update folder title length" 2024-09-11T07:58:26.736 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.498225703Z level=info msg="Executing migration" id="Add unique index for folder.title and folder.parent_uid" 2024-09-11T07:58:26.736 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=migrator t=2024-09-11T07:58:26.49875218Z level=info msg="migrations completed" performed=485 skipped=0 duration=718.355911ms 2024-09-11T07:58:26.736 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=sqlstore t=2024-09-11T07:58:26.499200743Z level=info msg="Created default organization" 2024-09-11T07:58:26.736 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=plugin.loader t=2024-09-11T07:58:26.558612349Z level=info msg="Plugin registered" pluginID=input 2024-09-11T07:58:26.736 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=plugin.loader t=2024-09-11T07:58:26.586829971Z level=info msg="Plugin registered" pluginID=grafana-piechart-panel 2024-09-11T07:58:26.737 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=plugin.loader t=2024-09-11T07:58:26.586856367Z level=info msg="Plugin registered" pluginID=vonage-status-panel 2024-09-11T07:58:26.737 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=secrets t=2024-09-11T07:58:26.586895947Z level=info msg="Envelope encryption state" enabled=true currentprovider=secretKey.v1 2024-09-11T07:58:26.737 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=query_data t=2024-09-11T07:58:26.588720328Z level=info msg="Query Service initialization" 2024-09-11T07:58:26.737 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=live.push_http t=2024-09-11T07:58:26.591363983Z level=info msg="Live Push Gateway initialization" 2024-09-11T07:58:27.570 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:27 smithi110 ceph-mon[31709]: Deploying daemon node-exporter.a on smithi110 2024-09-11T07:58:27.571 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:27 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:27.571 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:27 smithi110 ceph-mon[25392]: Deploying daemon node-exporter.a on smithi110 2024-09-11T07:58:27.571 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:27 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:27.626 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:27 smithi139 ceph-mon[27311]: Deploying daemon node-exporter.a on smithi110 2024-09-11T07:58:27.626 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:27 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:27.626 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:27 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=infra.usagestats.collector t=2024-09-11T07:58:27.375810893Z level=info msg="registering usage stat providers" usageStatsProvidersLen=2 2024-09-11T07:58:27.627 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:27 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=provisioning.datasources t=2024-09-11T07:58:27.376160565Z level=warn msg="[Deprecated] the datasource provisioning config is outdated. please upgrade" filename=/etc/grafana/provisioning/datasources/ceph-dashboard.yml 2024-09-11T07:58:27.627 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:27 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=provisioning.datasources t=2024-09-11T07:58:27.409096998Z level=info msg="inserting datasource from configuration " name=Dashboard1 uid=P43CA22E17D0F9596 2024-09-11T07:58:27.627 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:27 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=provisioning.datasources t=2024-09-11T07:58:27.440752919Z level=info msg="inserting datasource from configuration " name=Loki uid=P8E80F9AEF21F6940 2024-09-11T07:58:27.627 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:27 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=provisioning.alerting t=2024-09-11T07:58:27.463376969Z level=info msg="starting to provision alerting" 2024-09-11T07:58:27.627 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:27 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=provisioning.alerting t=2024-09-11T07:58:27.46339516Z level=info msg="finished to provision alerting" 2024-09-11T07:58:27.627 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:27 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=ngalert.state.manager t=2024-09-11T07:58:27.463541364Z level=info msg="Warming state cache for startup" 2024-09-11T07:58:27.627 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:27 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=grafanaStorageLogger t=2024-09-11T07:58:27.463610224Z level=info msg="storage starting" 2024-09-11T07:58:27.627 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:27 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=http.server t=2024-09-11T07:58:27.46524179Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=https subUrl= socket= 2024-09-11T07:58:27.627 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:27 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=ngalert.state.manager t=2024-09-11T07:58:27.47380367Z level=info msg="State cache has been initialized" states=0 duration=10.261462ms 2024-09-11T07:58:27.627 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:27 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=sqlstore.transactions t=2024-09-11T07:58:27.473835457Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-11T07:58:27.627 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:27 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=ticker t=2024-09-11T07:58:27.473864898Z level=info msg=starting first_tick=2024-09-11T07:58:30Z 2024-09-11T07:58:27.627 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:27 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=ngalert.multiorg.alertmanager t=2024-09-11T07:58:27.473888584Z level=info msg="starting MultiOrg Alertmanager" 2024-09-11T07:58:27.627 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:27 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=sqlstore.transactions t=2024-09-11T07:58:27.506715751Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-11T07:58:27.628 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:27 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=sqlstore.transactions t=2024-09-11T07:58:27.547560739Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-11T07:58:27.628 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 07:58:27 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a[51035]: logger=sqlstore.transactions t=2024-09-11T07:58:27.558028007Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=1 code="database is locked" 2024-09-11T07:58:28.052 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:58:28.199 INFO:teuthology.orchestra.run.smithi110.stdout:128849018915 2024-09-11T07:58:28.249 INFO:journalctl@ceph.iscsi.iscsi.a.smithi139.stdout:Sep 11 07:58:27 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-iscsi-iscsi-a[46755]: debug there is no tcmu-runner data available 2024-09-11T07:58:28.370 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:28 smithi110 ceph-mon[25392]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:58:28.370 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:28 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:28.371 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:28 smithi110 ceph-mon[25392]: from='client.? 172.21.15.110:0/2950173390' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-11T07:58:28.371 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:28 smithi110 ceph-mon[31709]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:58:28.371 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:28 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:28.371 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:28 smithi110 ceph-mon[31709]: from='client.? 172.21.15.110:0/2950173390' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-11T07:58:28.694 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 07:58:28 smithi110 systemd[1]: Starting Ceph node-exporter.a for 41bd1096-7012-11ef-bcea-c7b262605968... 2024-09-11T07:58:28.748 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:28 smithi139 ceph-mon[27311]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:58:28.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:28 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:28.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:28 smithi139 ceph-mon[27311]: from='client.? 172.21.15.110:0/2950173390' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-11T07:58:28.904 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574893 got 55834574898 for osd.1 2024-09-11T07:58:28.904 DEBUG:teuthology.parallel:result is None 2024-09-11T07:58:29.569 INFO:teuthology.orchestra.run.smithi110.stdout:163208757280 2024-09-11T07:58:29.569 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:29 smithi110 ceph-mon[31709]: from='client.24449 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:58:29.570 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:29 smithi110 ceph-mon[25392]: from='client.24449 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:58:29.570 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 07:58:29 smithi110 bash[57635]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-09-11T07:58:29.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:29 smithi139 ceph-mon[27311]: from='client.24449 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:58:29.968 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 07:58:29 smithi110 bash[57635]: Getting image source signatures 2024-09-11T07:58:29.968 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 07:58:29 smithi110 bash[57635]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-09-11T07:58:29.968 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 07:58:29 smithi110 bash[57635]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-09-11T07:58:29.968 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 07:58:29 smithi110 bash[57635]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-09-11T07:58:30.468 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:30 smithi110 ceph-mon[31709]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:58:30.468 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:30 smithi110 ceph-mon[31709]: from='client.? 172.21.15.110:0/208785171' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-11T07:58:30.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:30 smithi110 ceph-mon[25392]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:58:30.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:30 smithi110 ceph-mon[25392]: from='client.? 172.21.15.110:0/208785171' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-11T07:58:30.602 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:30 smithi139 ceph-mon[27311]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:58:30.602 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:30 smithi139 ceph-mon[27311]: from='client.? 172.21.15.110:0/208785171' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-11T07:58:31.301 INFO:teuthology.orchestra.run.smithi110.stdout:77309411375 2024-09-11T07:58:31.552 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 07:58:31 smithi110 bash[57635]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-09-11T07:58:31.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:31 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:31.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:31 smithi139 ceph-mon[27311]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:58:31.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:31 smithi139 ceph-mon[27311]: from='client.? 172.21.15.110:0/2684020548' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-11T07:58:31.967 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 07:58:31 smithi110 bash[57635]: Writing manifest to image destination 2024-09-11T07:58:31.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:31 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:31.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:31 smithi110 ceph-mon[25392]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:58:31.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:31 smithi110 ceph-mon[25392]: from='client.? 172.21.15.110:0/2684020548' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-11T07:58:31.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:31 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:31.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:31 smithi110 ceph-mon[31709]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:58:31.969 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:31 smithi110 ceph-mon[31709]: from='client.? 172.21.15.110:0/2684020548' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-11T07:58:32.408 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 07:58:32 smithi110 podman[57635]: 2024-09-11 07:58:32.197135232 +0000 UTC m=+3.089432892 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-09-11T07:58:32.714 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757274 got 163208757280 for osd.5 2024-09-11T07:58:32.714 DEBUG:teuthology.parallel:result is None 2024-09-11T07:58:32.863 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738420 got 34359738422 for osd.0 2024-09-11T07:58:32.863 DEBUG:teuthology.parallel:result is None 2024-09-11T07:58:32.958 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:32 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:58:32.959 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:32 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:58:32.998 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:32 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:58:33.213 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 07:58:32 smithi110 podman[57635]: 2024-09-11 07:58:32.957958556 +0000 UTC m=+3.850256219 container create 8c994666edb7f8440c3afff4fba58d55c427e31c5396dfe8d79f21cdf2966119 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-11T07:58:33.347 INFO:teuthology.orchestra.run.smithi110.stdout:231928234006 2024-09-11T07:58:33.468 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 07:58:33 smithi110 podman[57635]: 2024-09-11 07:58:33.403667832 +0000 UTC m=+4.295965487 container init 8c994666edb7f8440c3afff4fba58d55c427e31c5396dfe8d79f21cdf2966119 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-11T07:58:33.468 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 07:58:33 smithi110 podman[57635]: 2024-09-11 07:58:33.407235106 +0000 UTC m=+4.299532762 container start 8c994666edb7f8440c3afff4fba58d55c427e31c5396dfe8d79f21cdf2966119 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-11T07:58:33.468 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 07:58:33 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-a[57846]: ts=2024-09-11T07:58:33.410Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-09-11T07:58:33.469 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 07:58:33 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-a[57846]: ts=2024-09-11T07:58:33.411Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-09-11T07:58:33.469 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 07:58:33 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-a[57846]: ts=2024-09-11T07:58:33.411Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-11T07:58:33.469 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 07:58:33 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-a[57846]: ts=2024-09-11T07:58:33.411Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-11T07:58:33.469 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 07:58:33 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-a[57846]: ts=2024-09-11T07:58:33.411Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-11T07:58:33.469 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 07:58:33 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-a[57846]: ts=2024-09-11T07:58:33.411Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-11T07:58:33.469 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 07:58:33 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-a[57846]: ts=2024-09-11T07:58:33.412Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-11T07:58:33.469 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 07:58:33 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-a[57846]: ts=2024-09-11T07:58:33.412Z caller=node_exporter.go:117 level=info collector=arp 2024-09-11T07:58:33.469 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 07:58:33 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-a[57846]: ts=2024-09-11T07:58:33.412Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-11T07:58:33.469 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 07:58:33 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-a[57846]: ts=2024-09-11T07:58:33.412Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-11T07:58:33.469 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 07:58:33 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-a[57846]: ts=2024-09-11T07:58:33.412Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-11T07:58:33.469 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 07:58:33 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-a[57846]: ts=2024-09-11T07:58:33.412Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-11T07:58:33.469 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 07:58:33 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-a[57846]: ts=2024-09-11T07:58:33.412Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-11T07:58:33.469 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 07:58:33 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-a[57846]: ts=2024-09-11T07:58:33.412Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-11T07:58:33.470 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 07:58:33 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-a[57846]: ts=2024-09-11T07:58:33.412Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-11T07:58:33.470 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 07:58:33 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-a[57846]: ts=2024-09-11T07:58:33.412Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-11T07:58:33.470 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 07:58:33 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-a[57846]: ts=2024-09-11T07:58:33.412Z caller=node_exporter.go:117 level=info collector=edac 2024-09-11T07:58:33.470 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 07:58:33 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-a[57846]: ts=2024-09-11T07:58:33.412Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-11T07:58:33.470 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 07:58:33 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-a[57846]: ts=2024-09-11T07:58:33.412Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-11T07:58:33.470 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 07:58:33 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-a[57846]: ts=2024-09-11T07:58:33.412Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-11T07:58:33.470 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 07:58:33 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-a[57846]: ts=2024-09-11T07:58:33.412Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-11T07:58:33.470 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 07:58:33 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-a[57846]: ts=2024-09-11T07:58:33.412Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-11T07:58:33.471 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 07:58:33 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-a[57846]: ts=2024-09-11T07:58:33.412Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-11T07:58:33.471 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 07:58:33 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-a[57846]: ts=2024-09-11T07:58:33.412Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-11T07:58:33.471 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 07:58:33 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-a[57846]: ts=2024-09-11T07:58:33.412Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-11T07:58:33.471 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 07:58:33 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-a[57846]: ts=2024-09-11T07:58:33.412Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-11T07:58:33.471 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 07:58:33 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-a[57846]: ts=2024-09-11T07:58:33.412Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-11T07:58:33.471 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 07:58:33 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-a[57846]: ts=2024-09-11T07:58:33.412Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-11T07:58:33.471 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 07:58:33 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-a[57846]: ts=2024-09-11T07:58:33.412Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-11T07:58:33.471 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 07:58:33 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-a[57846]: ts=2024-09-11T07:58:33.412Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-11T07:58:33.471 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 07:58:33 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-a[57846]: ts=2024-09-11T07:58:33.412Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-11T07:58:33.471 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 07:58:33 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-a[57846]: ts=2024-09-11T07:58:33.412Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-11T07:58:33.471 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 07:58:33 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-a[57846]: ts=2024-09-11T07:58:33.412Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-11T07:58:33.472 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 07:58:33 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-a[57846]: ts=2024-09-11T07:58:33.412Z caller=node_exporter.go:117 level=info collector=os 2024-09-11T07:58:33.472 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 07:58:33 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-a[57846]: ts=2024-09-11T07:58:33.412Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-11T07:58:33.472 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 07:58:33 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-a[57846]: ts=2024-09-11T07:58:33.412Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-11T07:58:33.472 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 07:58:33 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-a[57846]: ts=2024-09-11T07:58:33.412Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-11T07:58:33.472 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 07:58:33 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-a[57846]: ts=2024-09-11T07:58:33.412Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-11T07:58:33.472 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 07:58:33 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-a[57846]: ts=2024-09-11T07:58:33.412Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-11T07:58:33.472 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 07:58:33 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-a[57846]: ts=2024-09-11T07:58:33.412Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-11T07:58:33.472 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 07:58:33 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-a[57846]: ts=2024-09-11T07:58:33.412Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-11T07:58:33.472 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 07:58:33 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-a[57846]: ts=2024-09-11T07:58:33.412Z caller=node_exporter.go:117 level=info collector=stat 2024-09-11T07:58:33.472 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 07:58:33 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-a[57846]: ts=2024-09-11T07:58:33.412Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-11T07:58:33.472 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 07:58:33 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-a[57846]: ts=2024-09-11T07:58:33.412Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-11T07:58:33.472 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 07:58:33 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-a[57846]: ts=2024-09-11T07:58:33.412Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-11T07:58:33.473 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 07:58:33 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-a[57846]: ts=2024-09-11T07:58:33.412Z caller=node_exporter.go:117 level=info collector=time 2024-09-11T07:58:33.473 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 07:58:33 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-a[57846]: ts=2024-09-11T07:58:33.412Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-11T07:58:33.473 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 07:58:33 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-a[57846]: ts=2024-09-11T07:58:33.412Z caller=node_exporter.go:117 level=info collector=uname 2024-09-11T07:58:33.473 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 07:58:33 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-a[57846]: ts=2024-09-11T07:58:33.412Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-11T07:58:33.473 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 07:58:33 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-a[57846]: ts=2024-09-11T07:58:33.412Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-11T07:58:33.473 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 07:58:33 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-a[57846]: ts=2024-09-11T07:58:33.412Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-11T07:58:33.473 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 07:58:33 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-a[57846]: ts=2024-09-11T07:58:33.412Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-09-11T07:58:33.473 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 07:58:33 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-a[57846]: ts=2024-09-11T07:58:33.412Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-11T07:58:33.748 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:33 smithi139 ceph-mon[27311]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:58:33.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:33 smithi139 ceph-mon[27311]: from='client.? 172.21.15.110:0/321514616' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-11T07:58:33.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:33 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:33.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:33 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:33.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:33 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:33.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:33 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:33.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:33 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:33.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:33 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:33.916 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 07:58:33 smithi110 bash[57635]: 8c994666edb7f8440c3afff4fba58d55c427e31c5396dfe8d79f21cdf2966119 2024-09-11T07:58:33.916 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 07:58:33 smithi110 systemd[1]: Started Ceph node-exporter.a for 41bd1096-7012-11ef-bcea-c7b262605968. 2024-09-11T07:58:33.917 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:33 smithi110 ceph-mon[31709]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:58:33.917 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:33 smithi110 ceph-mon[31709]: from='client.? 172.21.15.110:0/321514616' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-11T07:58:33.917 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:33 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:33.917 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:33 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:33.917 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:33 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:33.917 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:33 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:33.917 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:33 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:33.917 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:33 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:33.917 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:33 smithi110 ceph-mon[25392]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:58:33.917 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:33 smithi110 ceph-mon[25392]: from='client.? 172.21.15.110:0/321514616' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-11T07:58:33.918 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:33 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:33.918 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:33 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:33.918 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:33 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:33.918 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:33 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:33.918 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:33 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:33.918 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:33 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:33.949 INFO:tasks.cephadm.ceph_manager.ceph:need seq 128849018911 got 128849018915 for osd.4 2024-09-11T07:58:33.949 DEBUG:teuthology.parallel:result is None 2024-09-11T07:58:34.123 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:58:33 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x[29520]: ::ffff:172.21.15.139 - - [11/Sep/2024:07:58:33] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:58:34.499 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 07:58:34 smithi139 systemd[1]: Starting Ceph node-exporter.b for 41bd1096-7012-11ef-bcea-c7b262605968... 2024-09-11T07:58:34.853 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411371 got 77309411375 for osd.2 2024-09-11T07:58:34.854 DEBUG:teuthology.parallel:result is None 2024-09-11T07:58:34.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:34 smithi139 ceph-mon[27311]: Deploying daemon node-exporter.b on smithi139 2024-09-11T07:58:34.858 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 07:58:34 smithi139 bash[52066]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-09-11T07:58:35.089 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:34 smithi110 ceph-mon[25392]: Deploying daemon node-exporter.b on smithi139 2024-09-11T07:58:35.089 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:34 smithi110 ceph-mon[31709]: Deploying daemon node-exporter.b on smithi139 2024-09-11T07:58:35.248 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 07:58:34 smithi139 bash[52066]: Getting image source signatures 2024-09-11T07:58:35.248 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 07:58:34 smithi139 bash[52066]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-09-11T07:58:35.249 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 07:58:34 smithi139 bash[52066]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-09-11T07:58:35.249 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 07:58:34 smithi139 bash[52066]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-09-11T07:58:35.757 INFO:tasks.cephadm.ceph_manager.ceph:need seq 231928234002 got 231928234006 for osd.7 2024-09-11T07:58:35.757 DEBUG:teuthology.parallel:result is None 2024-09-11T07:58:35.992 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:35 smithi110 ceph-mon[31709]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:58:35.992 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:35 smithi110 ceph-mon[25392]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:58:35.998 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:35 smithi139 ceph-mon[27311]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:58:36.498 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 07:58:36 smithi139 bash[52066]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-09-11T07:58:36.498 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 07:58:36 smithi139 bash[52066]: Writing manifest to image destination 2024-09-11T07:58:36.860 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 07:58:36 smithi139 podman[52066]: 2024-09-11 07:58:36.542372448 +0000 UTC m=+2.117202575 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-09-11T07:58:36.860 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 07:58:36 smithi139 podman[52066]: 2024-09-11 07:58:36.657482788 +0000 UTC m=+2.232312954 container create 82df887f4df0ddc1d40364fc6b65894d6136d08040e45743647a366736d3e37b (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-11T07:58:37.249 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 07:58:36 smithi139 podman[52066]: 2024-09-11 07:58:36.859593047 +0000 UTC m=+2.434423178 container init 82df887f4df0ddc1d40364fc6b65894d6136d08040e45743647a366736d3e37b (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-11T07:58:37.249 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 07:58:36 smithi139 podman[52066]: 2024-09-11 07:58:36.862574864 +0000 UTC m=+2.437404991 container start 82df887f4df0ddc1d40364fc6b65894d6136d08040e45743647a366736d3e37b (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-11T07:58:37.249 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 07:58:36 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-b[52143]: ts=2024-09-11T07:58:36.866Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-09-11T07:58:37.249 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 07:58:36 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-b[52143]: ts=2024-09-11T07:58:36.866Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-09-11T07:58:37.250 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 07:58:36 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-b[52143]: ts=2024-09-11T07:58:36.866Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-11T07:58:37.250 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 07:58:36 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-b[52143]: ts=2024-09-11T07:58:36.866Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-11T07:58:37.250 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 07:58:36 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-b[52143]: ts=2024-09-11T07:58:36.866Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-11T07:58:37.250 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 07:58:36 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-b[52143]: ts=2024-09-11T07:58:36.866Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-11T07:58:37.250 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 07:58:36 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-b[52143]: ts=2024-09-11T07:58:36.867Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-11T07:58:37.250 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 07:58:36 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-b[52143]: ts=2024-09-11T07:58:36.867Z caller=node_exporter.go:117 level=info collector=arp 2024-09-11T07:58:37.250 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 07:58:36 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-b[52143]: ts=2024-09-11T07:58:36.867Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-11T07:58:37.250 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 07:58:36 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-b[52143]: ts=2024-09-11T07:58:36.867Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-11T07:58:37.250 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 07:58:36 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-b[52143]: ts=2024-09-11T07:58:36.867Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-11T07:58:37.250 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 07:58:36 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-b[52143]: ts=2024-09-11T07:58:36.867Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-11T07:58:37.250 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 07:58:36 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-b[52143]: ts=2024-09-11T07:58:36.867Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-11T07:58:37.250 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 07:58:36 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-b[52143]: ts=2024-09-11T07:58:36.867Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-11T07:58:37.250 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 07:58:36 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-b[52143]: ts=2024-09-11T07:58:36.867Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-11T07:58:37.250 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 07:58:36 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-b[52143]: ts=2024-09-11T07:58:36.867Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-11T07:58:37.251 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 07:58:36 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-b[52143]: ts=2024-09-11T07:58:36.867Z caller=node_exporter.go:117 level=info collector=edac 2024-09-11T07:58:37.251 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 07:58:36 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-b[52143]: ts=2024-09-11T07:58:36.867Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-11T07:58:37.251 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 07:58:36 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-b[52143]: ts=2024-09-11T07:58:36.867Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-11T07:58:37.251 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 07:58:36 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-b[52143]: ts=2024-09-11T07:58:36.867Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-11T07:58:37.251 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 07:58:36 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-b[52143]: ts=2024-09-11T07:58:36.867Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-11T07:58:37.251 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 07:58:36 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-b[52143]: ts=2024-09-11T07:58:36.867Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-11T07:58:37.251 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 07:58:36 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-b[52143]: ts=2024-09-11T07:58:36.867Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-11T07:58:37.251 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 07:58:36 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-b[52143]: ts=2024-09-11T07:58:36.867Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-11T07:58:37.251 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 07:58:36 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-b[52143]: ts=2024-09-11T07:58:36.867Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-11T07:58:37.251 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 07:58:36 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-b[52143]: ts=2024-09-11T07:58:36.867Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-11T07:58:37.251 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 07:58:36 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-b[52143]: ts=2024-09-11T07:58:36.867Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-11T07:58:37.251 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 07:58:36 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-b[52143]: ts=2024-09-11T07:58:36.867Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-11T07:58:37.251 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 07:58:36 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-b[52143]: ts=2024-09-11T07:58:36.867Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-11T07:58:37.252 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 07:58:36 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-b[52143]: ts=2024-09-11T07:58:36.867Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-11T07:58:37.252 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 07:58:36 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-b[52143]: ts=2024-09-11T07:58:36.867Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-11T07:58:37.252 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 07:58:36 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-b[52143]: ts=2024-09-11T07:58:36.867Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-11T07:58:37.252 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 07:58:36 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-b[52143]: ts=2024-09-11T07:58:36.867Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-11T07:58:37.252 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 07:58:36 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-b[52143]: ts=2024-09-11T07:58:36.867Z caller=node_exporter.go:117 level=info collector=os 2024-09-11T07:58:37.252 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 07:58:36 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-b[52143]: ts=2024-09-11T07:58:36.867Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-11T07:58:37.252 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 07:58:36 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-b[52143]: ts=2024-09-11T07:58:36.867Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-11T07:58:37.252 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 07:58:36 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-b[52143]: ts=2024-09-11T07:58:36.867Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-11T07:58:37.252 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 07:58:36 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-b[52143]: ts=2024-09-11T07:58:36.867Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-11T07:58:37.253 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 07:58:36 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-b[52143]: ts=2024-09-11T07:58:36.867Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-11T07:58:37.253 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 07:58:36 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-b[52143]: ts=2024-09-11T07:58:36.868Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-11T07:58:37.253 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 07:58:36 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-b[52143]: ts=2024-09-11T07:58:36.868Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-11T07:58:37.253 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 07:58:36 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-b[52143]: ts=2024-09-11T07:58:36.868Z caller=node_exporter.go:117 level=info collector=stat 2024-09-11T07:58:37.253 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 07:58:36 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-b[52143]: ts=2024-09-11T07:58:36.868Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-11T07:58:37.253 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 07:58:36 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-b[52143]: ts=2024-09-11T07:58:36.868Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-11T07:58:37.253 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 07:58:36 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-b[52143]: ts=2024-09-11T07:58:36.868Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-11T07:58:37.253 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 07:58:36 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-b[52143]: ts=2024-09-11T07:58:36.868Z caller=node_exporter.go:117 level=info collector=time 2024-09-11T07:58:37.253 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 07:58:36 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-b[52143]: ts=2024-09-11T07:58:36.868Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-11T07:58:37.253 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 07:58:36 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-b[52143]: ts=2024-09-11T07:58:36.868Z caller=node_exporter.go:117 level=info collector=uname 2024-09-11T07:58:37.253 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 07:58:36 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-b[52143]: ts=2024-09-11T07:58:36.868Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-11T07:58:37.253 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 07:58:36 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-b[52143]: ts=2024-09-11T07:58:36.868Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-11T07:58:37.253 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 07:58:36 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-b[52143]: ts=2024-09-11T07:58:36.868Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-11T07:58:37.253 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 07:58:36 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-b[52143]: ts=2024-09-11T07:58:36.868Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-09-11T07:58:37.254 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 07:58:36 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-b[52143]: ts=2024-09-11T07:58:36.868Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-11T07:58:37.254 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 07:58:36 smithi139 bash[52066]: 82df887f4df0ddc1d40364fc6b65894d6136d08040e45743647a366736d3e37b 2024-09-11T07:58:37.254 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 07:58:36 smithi139 systemd[1]: Started Ceph node-exporter.b for 41bd1096-7012-11ef-bcea-c7b262605968. 2024-09-11T07:58:37.706 INFO:teuthology.orchestra.run.smithi110.stdout:107374182443 2024-09-11T07:58:38.249 INFO:journalctl@ceph.iscsi.iscsi.a.smithi139.stdout:Sep 11 07:58:37 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-iscsi-iscsi-a[46755]: debug there is no tcmu-runner data available 2024-09-11T07:58:38.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:38 smithi139 ceph-mon[27311]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:58:38.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:38 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:38.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:38 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:38.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:38 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:38.250 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:38 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:38.250 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:38 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:38.250 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:38 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:38.250 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:38 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:58:38.250 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:38 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:58:38.250 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:38 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:58:38.250 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:38 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:38.250 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:38 smithi139 ceph-mon[27311]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-11T07:58:38.250 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:38 smithi139 ceph-mon[27311]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-11T07:58:38.250 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:38 smithi139 ceph-mon[27311]: Reconfiguring daemon alertmanager.a on smithi110 2024-09-11T07:58:38.250 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:38 smithi139 ceph-mon[27311]: from='client.? 172.21.15.110:0/411088688' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-11T07:58:38.268 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:38 smithi110 ceph-mon[31709]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:58:38.268 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:38 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:38.268 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:38 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:38.268 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:38 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:38.268 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:38 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:38.268 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:38 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:38.268 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:38 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:38.268 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:38 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:58:38.268 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:38 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:58:38.269 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:38 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:58:38.269 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:38 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:38.269 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:38 smithi110 ceph-mon[31709]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-11T07:58:38.269 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:38 smithi110 ceph-mon[31709]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-11T07:58:38.269 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:38 smithi110 ceph-mon[31709]: Reconfiguring daemon alertmanager.a on smithi110 2024-09-11T07:58:38.269 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:38 smithi110 ceph-mon[31709]: from='client.? 172.21.15.110:0/411088688' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-11T07:58:38.269 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:38 smithi110 ceph-mon[25392]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:58:38.269 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:38 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:38.269 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:38 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:38.269 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:38 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:38.269 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:38 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:38.269 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:38 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:38.270 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:38 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:38.270 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:38 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:58:38.270 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:38 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:58:38.270 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:38 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:58:38.270 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:38 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:38.270 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:38 smithi110 ceph-mon[25392]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-11T07:58:38.270 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:38 smithi110 ceph-mon[25392]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-11T07:58:38.270 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:38 smithi110 ceph-mon[25392]: Reconfiguring daemon alertmanager.a on smithi110 2024-09-11T07:58:38.270 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:38 smithi110 ceph-mon[25392]: from='client.? 172.21.15.110:0/411088688' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-11T07:58:39.169 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182438 got 107374182443 for osd.3 2024-09-11T07:58:39.169 DEBUG:teuthology.parallel:result is None 2024-09-11T07:58:39.169 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-11T07:58:39.170 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T07:58:39.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:39 smithi110 ceph-mon[25392]: from='client.24449 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:58:39.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:39 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:39.468 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:39 smithi110 ceph-mon[31709]: from='client.24449 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:58:39.468 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:39 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:39.494 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:39 smithi139 ceph-mon[27311]: from='client.24449 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:58:39.494 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:39 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:39.973 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:58:40.468 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:40 smithi110 ceph-mon[31709]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:58:40.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:40 smithi110 ceph-mon[25392]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:58:40.498 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:40 smithi139 ceph-mon[27311]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:58:42.339 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:42 smithi110 ceph-mon[31709]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:58:42.339 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:42 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:42.340 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:42 smithi110 ceph-mon[25392]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:58:42.340 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:42 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:42.391 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:42 smithi139 ceph-mon[27311]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:58:42.392 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:42 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:42.593 INFO:journalctl@ceph.alertmanager.a.smithi110.stdout:Sep 11 07:58:42 smithi110 systemd[1]: Stopping Ceph alertmanager.a for 41bd1096-7012-11ef-bcea-c7b262605968... 2024-09-11T07:58:43.941 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:58:43 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x[29520]: ::ffff:172.21.15.139 - - [11/Sep/2024:07:58:43] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:58:44.173 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T07:58:44.175 INFO:teuthology.orchestra.run.smithi110.stderr:dumped all 2024-09-11T07:58:44.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:44 smithi139 ceph-mon[27311]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:58:44.460 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:44 smithi110 ceph-mon[25392]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:58:44.460 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:44 smithi110 ceph-mon[31709]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:58:45.486 INFO:teuthology.orchestra.run.smithi110.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2024-09-11T07:58:42.965285+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596188,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":994,"num_read_kb":847,"num_write":609,"num_write_kb":3237,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":1771682,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":635,"ondisk_log_size":635,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749928448,"kb_used":2386776,"kb_used_data":7936,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":747541672,"statfs":{"total":767926730752,"available":765482672128,"internally_reserved":0,"allocated":8126464,"data_stored":4425351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003098"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.572715+0000","last_change":"2024-09-11T07:57:10.635500+0000","last_active":"2024-09-11T07:57:33.572715+0000","last_peered":"2024-09-11T07:57:33.572715+0000","last_clean":"2024-09-11T07:57:33.572715+0000","last_became_active":"2024-09-11T07:57:10.635199+0000","last_became_peered":"2024-09-11T07:57:10.635199+0000","last_unstale":"2024-09-11T07:57:33.572715+0000","last_undegraded":"2024-09-11T07:57:33.572715+0000","last_fullsized":"2024-09-11T07:57:33.572715+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_clean_scrub_stamp":"2024-09-11T07:57:09.499431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:06.145642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"60'1","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.611923+0000","last_change":"2024-09-11T07:57:04.484423+0000","last_active":"2024-09-11T07:57:33.611923+0000","last_peered":"2024-09-11T07:57:33.611923+0000","last_clean":"2024-09-11T07:57:33.611923+0000","last_became_active":"2024-09-11T07:57:04.484049+0000","last_became_peered":"2024-09-11T07:57:04.484049+0000","last_unstale":"2024-09-11T07:57:33.611923+0000","last_undegraded":"2024-09-11T07:57:33.611923+0000","last_fullsized":"2024-09-11T07:57:33.611923+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:15:21.924817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"67'15","reported_seq":43,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.572935+0000","last_change":"2024-09-11T07:57:06.492851+0000","last_active":"2024-09-11T07:57:33.572935+0000","last_peered":"2024-09-11T07:57:33.572935+0000","last_clean":"2024-09-11T07:57:33.572935+0000","last_became_active":"2024-09-11T07:57:06.492565+0000","last_became_peered":"2024-09-11T07:57:06.492565+0000","last_unstale":"2024-09-11T07:57:33.572935+0000","last_undegraded":"2024-09-11T07:57:33.572935+0000","last_fullsized":"2024-09-11T07:57:33.572935+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_clean_scrub_stamp":"2024-09-11T07:57:05.472156+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:43:59.640735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.578960+0000","last_change":"2024-09-11T07:57:08.508110+0000","last_active":"2024-09-11T07:57:33.578960+0000","last_peered":"2024-09-11T07:57:33.578960+0000","last_clean":"2024-09-11T07:57:33.578960+0000","last_became_active":"2024-09-11T07:57:08.507536+0000","last_became_peered":"2024-09-11T07:57:08.507536+0000","last_unstale":"2024-09-11T07:57:33.578960+0000","last_undegraded":"2024-09-11T07:57:33.578960+0000","last_fullsized":"2024-09-11T07:57:33.578960+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_clean_scrub_stamp":"2024-09-11T07:57:07.482151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:17:21.043646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.574406+0000","last_change":"2024-09-11T07:57:04.485662+0000","last_active":"2024-09-11T07:57:33.574406+0000","last_peered":"2024-09-11T07:57:33.574406+0000","last_clean":"2024-09-11T07:57:33.574406+0000","last_became_active":"2024-09-11T07:57:04.485413+0000","last_became_peered":"2024-09-11T07:57:04.485413+0000","last_unstale":"2024-09-11T07:57:33.574406+0000","last_undegraded":"2024-09-11T07:57:33.574406+0000","last_fullsized":"2024-09-11T07:57:33.574406+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:10.763604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"67'9","reported_seq":34,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.578917+0000","last_change":"2024-09-11T07:57:06.498071+0000","last_active":"2024-09-11T07:57:33.578917+0000","last_peered":"2024-09-11T07:57:33.578917+0000","last_clean":"2024-09-11T07:57:33.578917+0000","last_became_active":"2024-09-11T07:57:06.497363+0000","last_became_peered":"2024-09-11T07:57:06.497363+0000","last_unstale":"2024-09-11T07:57:33.578917+0000","last_undegraded":"2024-09-11T07:57:33.578917+0000","last_fullsized":"2024-09-11T07:57:33.578917+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_clean_scrub_stamp":"2024-09-11T07:57:05.472156+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:00:51.725341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.573195+0000","last_change":"2024-09-11T07:57:08.512337+0000","last_active":"2024-09-11T07:57:33.573195+0000","last_peered":"2024-09-11T07:57:33.573195+0000","last_clean":"2024-09-11T07:57:33.573195+0000","last_became_active":"2024-09-11T07:57:08.512058+0000","last_became_peered":"2024-09-11T07:57:08.512058+0000","last_unstale":"2024-09-11T07:57:33.573195+0000","last_undegraded":"2024-09-11T07:57:33.573195+0000","last_fullsized":"2024-09-11T07:57:33.573195+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_clean_scrub_stamp":"2024-09-11T07:57:07.482151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:20:26.339645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.579191+0000","last_change":"2024-09-11T07:57:10.517102+0000","last_active":"2024-09-11T07:57:33.579191+0000","last_peered":"2024-09-11T07:57:33.579191+0000","last_clean":"2024-09-11T07:57:33.579191+0000","last_became_active":"2024-09-11T07:57:10.516696+0000","last_became_peered":"2024-09-11T07:57:10.516696+0000","last_unstale":"2024-09-11T07:57:33.579191+0000","last_undegraded":"2024-09-11T07:57:33.579191+0000","last_fullsized":"2024-09-11T07:57:33.579191+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_clean_scrub_stamp":"2024-09-11T07:57:09.499431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:41:18.372163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.573887+0000","last_change":"2024-09-11T07:57:04.491475+0000","last_active":"2024-09-11T07:57:33.573887+0000","last_peered":"2024-09-11T07:57:33.573887+0000","last_clean":"2024-09-11T07:57:33.573887+0000","last_became_active":"2024-09-11T07:57:04.491229+0000","last_became_peered":"2024-09-11T07:57:04.491229+0000","last_unstale":"2024-09-11T07:57:33.573887+0000","last_undegraded":"2024-09-11T07:57:33.573887+0000","last_fullsized":"2024-09-11T07:57:33.573887+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:29:14.948203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"67'5","reported_seq":28,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.579305+0000","last_change":"2024-09-11T07:57:06.499626+0000","last_active":"2024-09-11T07:57:33.579305+0000","last_peered":"2024-09-11T07:57:33.579305+0000","last_clean":"2024-09-11T07:57:33.579305+0000","last_became_active":"2024-09-11T07:57:06.499336+0000","last_became_peered":"2024-09-11T07:57:06.499336+0000","last_unstale":"2024-09-11T07:57:33.579305+0000","last_undegraded":"2024-09-11T07:57:33.579305+0000","last_fullsized":"2024-09-11T07:57:33.579305+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_clean_scrub_stamp":"2024-09-11T07:57:05.472156+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:37:43.301012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.573106+0000","last_change":"2024-09-11T07:57:08.511340+0000","last_active":"2024-09-11T07:57:33.573106+0000","last_peered":"2024-09-11T07:57:33.573106+0000","last_clean":"2024-09-11T07:57:33.573106+0000","last_became_active":"2024-09-11T07:57:08.511070+0000","last_became_peered":"2024-09-11T07:57:08.511070+0000","last_unstale":"2024-09-11T07:57:33.573106+0000","last_undegraded":"2024-09-11T07:57:33.573106+0000","last_fullsized":"2024-09-11T07:57:33.573106+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_clean_scrub_stamp":"2024-09-11T07:57:07.482151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:28:53.140985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.576117+0000","last_change":"2024-09-11T07:57:10.516638+0000","last_active":"2024-09-11T07:57:33.576117+0000","last_peered":"2024-09-11T07:57:33.576117+0000","last_clean":"2024-09-11T07:57:33.576117+0000","last_became_active":"2024-09-11T07:57:10.516359+0000","last_became_peered":"2024-09-11T07:57:10.516359+0000","last_unstale":"2024-09-11T07:57:33.576117+0000","last_undegraded":"2024-09-11T07:57:33.576117+0000","last_fullsized":"2024-09-11T07:57:33.576117+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_clean_scrub_stamp":"2024-09-11T07:57:09.499431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:21:21.854315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.573887+0000","last_change":"2024-09-11T07:57:04.490240+0000","last_active":"2024-09-11T07:57:33.573887+0000","last_peered":"2024-09-11T07:57:33.573887+0000","last_clean":"2024-09-11T07:57:33.573887+0000","last_became_active":"2024-09-11T07:57:04.489947+0000","last_became_peered":"2024-09-11T07:57:04.489947+0000","last_unstale":"2024-09-11T07:57:33.573887+0000","last_undegraded":"2024-09-11T07:57:33.573887+0000","last_fullsized":"2024-09-11T07:57:33.573887+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:44:17.845923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"67'9","reported_seq":34,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.579540+0000","last_change":"2024-09-11T07:57:06.499163+0000","last_active":"2024-09-11T07:57:33.579540+0000","last_peered":"2024-09-11T07:57:33.579540+0000","last_clean":"2024-09-11T07:57:33.579540+0000","last_became_active":"2024-09-11T07:57:06.498879+0000","last_became_peered":"2024-09-11T07:57:06.498879+0000","last_unstale":"2024-09-11T07:57:33.579540+0000","last_undegraded":"2024-09-11T07:57:33.579540+0000","last_fullsized":"2024-09-11T07:57:33.579540+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_clean_scrub_stamp":"2024-09-11T07:57:05.472156+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:30:26.594241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.576341+0000","last_change":"2024-09-11T07:57:08.507054+0000","last_active":"2024-09-11T07:57:33.576341+0000","last_peered":"2024-09-11T07:57:33.576341+0000","last_clean":"2024-09-11T07:57:33.576341+0000","last_became_active":"2024-09-11T07:57:08.506736+0000","last_became_peered":"2024-09-11T07:57:08.506736+0000","last_unstale":"2024-09-11T07:57:33.576341+0000","last_undegraded":"2024-09-11T07:57:33.576341+0000","last_fullsized":"2024-09-11T07:57:33.576341+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_clean_scrub_stamp":"2024-09-11T07:57:07.482151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:27:04.757009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.611333+0000","last_change":"2024-09-11T07:57:10.519267+0000","last_active":"2024-09-11T07:57:33.611333+0000","last_peered":"2024-09-11T07:57:33.611333+0000","last_clean":"2024-09-11T07:57:33.611333+0000","last_became_active":"2024-09-11T07:57:10.518870+0000","last_became_peered":"2024-09-11T07:57:10.518870+0000","last_unstale":"2024-09-11T07:57:33.611333+0000","last_undegraded":"2024-09-11T07:57:33.611333+0000","last_fullsized":"2024-09-11T07:57:33.611333+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_clean_scrub_stamp":"2024-09-11T07:57:09.499431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:30:22.260019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"67'17","reported_seq":46,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.579873+0000","last_change":"2024-09-11T07:57:06.493823+0000","last_active":"2024-09-11T07:57:33.579873+0000","last_peered":"2024-09-11T07:57:33.579873+0000","last_clean":"2024-09-11T07:57:33.579873+0000","last_became_active":"2024-09-11T07:57:06.493511+0000","last_became_peered":"2024-09-11T07:57:06.493511+0000","last_unstale":"2024-09-11T07:57:33.579873+0000","last_undegraded":"2024-09-11T07:57:33.579873+0000","last_fullsized":"2024-09-11T07:57:33.579873+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_clean_scrub_stamp":"2024-09-11T07:57:05.472156+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:16:59.506222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.573864+0000","last_change":"2024-09-11T07:57:04.489834+0000","last_active":"2024-09-11T07:57:33.573864+0000","last_peered":"2024-09-11T07:57:33.573864+0000","last_clean":"2024-09-11T07:57:33.573864+0000","last_became_active":"2024-09-11T07:57:04.489555+0000","last_became_peered":"2024-09-11T07:57:04.489555+0000","last_unstale":"2024-09-11T07:57:33.573864+0000","last_undegraded":"2024-09-11T07:57:33.573864+0000","last_fullsized":"2024-09-11T07:57:33.573864+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:45:20.906450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.573594+0000","last_change":"2024-09-11T07:57:08.510256+0000","last_active":"2024-09-11T07:57:33.573594+0000","last_peered":"2024-09-11T07:57:33.573594+0000","last_clean":"2024-09-11T07:57:33.573594+0000","last_became_active":"2024-09-11T07:57:08.509077+0000","last_became_peered":"2024-09-11T07:57:08.509077+0000","last_unstale":"2024-09-11T07:57:33.573594+0000","last_undegraded":"2024-09-11T07:57:33.573594+0000","last_fullsized":"2024-09-11T07:57:33.573594+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_clean_scrub_stamp":"2024-09-11T07:57:07.482151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:40:02.315603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.578928+0000","last_change":"2024-09-11T07:57:10.517096+0000","last_active":"2024-09-11T07:57:33.578928+0000","last_peered":"2024-09-11T07:57:33.578928+0000","last_clean":"2024-09-11T07:57:33.578928+0000","last_became_active":"2024-09-11T07:57:10.516818+0000","last_became_peered":"2024-09-11T07:57:10.516818+0000","last_unstale":"2024-09-11T07:57:33.578928+0000","last_undegraded":"2024-09-11T07:57:33.578928+0000","last_fullsized":"2024-09-11T07:57:33.578928+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_clean_scrub_stamp":"2024-09-11T07:57:09.499431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:11:14.042980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"67'10","reported_seq":33,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.579704+0000","last_change":"2024-09-11T07:57:06.498298+0000","last_active":"2024-09-11T07:57:33.579704+0000","last_peered":"2024-09-11T07:57:33.579704+0000","last_clean":"2024-09-11T07:57:33.579704+0000","last_became_active":"2024-09-11T07:57:06.498028+0000","last_became_peered":"2024-09-11T07:57:06.498028+0000","last_unstale":"2024-09-11T07:57:33.579704+0000","last_undegraded":"2024-09-11T07:57:33.579704+0000","last_fullsized":"2024-09-11T07:57:33.579704+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_clean_scrub_stamp":"2024-09-11T07:57:05.472156+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:50:04.600879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.574631+0000","last_change":"2024-09-11T07:57:04.482215+0000","last_active":"2024-09-11T07:57:33.574631+0000","last_peered":"2024-09-11T07:57:33.574631+0000","last_clean":"2024-09-11T07:57:33.574631+0000","last_became_active":"2024-09-11T07:57:04.481911+0000","last_became_peered":"2024-09-11T07:57:04.481911+0000","last_unstale":"2024-09-11T07:57:33.574631+0000","last_undegraded":"2024-09-11T07:57:33.574631+0000","last_fullsized":"2024-09-11T07:57:33.574631+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:52:00.542320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"67'11","reported_seq":47,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-11T07:58:37.535384+0000","last_change":"2024-09-11T07:57:08.504883+0000","last_active":"2024-09-11T07:58:37.535384+0000","last_peered":"2024-09-11T07:58:37.535384+0000","last_clean":"2024-09-11T07:58:37.535384+0000","last_became_active":"2024-09-11T07:57:08.504583+0000","last_became_peered":"2024-09-11T07:57:08.504583+0000","last_unstale":"2024-09-11T07:58:37.535384+0000","last_undegraded":"2024-09-11T07:58:37.535384+0000","last_fullsized":"2024-09-11T07:58:37.535384+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_clean_scrub_stamp":"2024-09-11T07:57:07.482151+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:23:43.798804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.579696+0000","last_change":"2024-09-11T07:57:10.512040+0000","last_active":"2024-09-11T07:57:33.579696+0000","last_peered":"2024-09-11T07:57:33.579696+0000","last_clean":"2024-09-11T07:57:33.579696+0000","last_became_active":"2024-09-11T07:57:10.511769+0000","last_became_peered":"2024-09-11T07:57:10.511769+0000","last_unstale":"2024-09-11T07:57:33.579696+0000","last_undegraded":"2024-09-11T07:57:33.579696+0000","last_fullsized":"2024-09-11T07:57:33.579696+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_clean_scrub_stamp":"2024-09-11T07:57:09.499431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:08:14.606379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"67'15","reported_seq":27,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.606245+0000","last_change":"2024-09-11T07:57:33.606010+0000","last_active":"2024-09-11T07:57:33.606245+0000","last_peered":"2024-09-11T07:57:33.606245+0000","last_clean":"2024-09-11T07:57:33.606245+0000","last_became_active":"2024-09-11T07:57:33.580137+0000","last_became_peered":"2024-09-11T07:57:33.580137+0000","last_unstale":"2024-09-11T07:57:33.606245+0000","last_undegraded":"2024-09-11T07:57:33.606245+0000","last_fullsized":"2024-09-11T07:57:33.606245+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_clean_scrub_stamp":"2024-09-11T07:57:05.472156+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:22:44.605304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.573384+0000","last_change":"2024-09-11T07:57:04.481731+0000","last_active":"2024-09-11T07:57:33.573384+0000","last_peered":"2024-09-11T07:57:33.573384+0000","last_clean":"2024-09-11T07:57:33.573384+0000","last_became_active":"2024-09-11T07:57:04.481457+0000","last_became_peered":"2024-09-11T07:57:04.481457+0000","last_unstale":"2024-09-11T07:57:33.573384+0000","last_undegraded":"2024-09-11T07:57:33.573384+0000","last_fullsized":"2024-09-11T07:57:33.573384+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:53:09.159361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"67'11","reported_seq":47,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-11T07:58:37.535385+0000","last_change":"2024-09-11T07:57:08.503740+0000","last_active":"2024-09-11T07:58:37.535385+0000","last_peered":"2024-09-11T07:58:37.535385+0000","last_clean":"2024-09-11T07:58:37.535385+0000","last_became_active":"2024-09-11T07:57:08.503438+0000","last_became_peered":"2024-09-11T07:57:08.503438+0000","last_unstale":"2024-09-11T07:58:37.535385+0000","last_undegraded":"2024-09-11T07:58:37.535385+0000","last_fullsized":"2024-09-11T07:58:37.535385+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_clean_scrub_stamp":"2024-09-11T07:57:07.482151+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:36:10.473849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.576558+0000","last_change":"2024-09-11T07:57:10.516051+0000","last_active":"2024-09-11T07:57:33.576558+0000","last_peered":"2024-09-11T07:57:33.576558+0000","last_clean":"2024-09-11T07:57:33.576558+0000","last_became_active":"2024-09-11T07:57:10.515581+0000","last_became_peered":"2024-09-11T07:57:10.515581+0000","last_unstale":"2024-09-11T07:57:33.576558+0000","last_undegraded":"2024-09-11T07:57:33.576558+0000","last_fullsized":"2024-09-11T07:57:33.576558+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_clean_scrub_stamp":"2024-09-11T07:57:09.499431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:21:35.149214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"67'11","reported_seq":37,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.579803+0000","last_change":"2024-09-11T07:57:06.499592+0000","last_active":"2024-09-11T07:57:33.579803+0000","last_peered":"2024-09-11T07:57:33.579803+0000","last_clean":"2024-09-11T07:57:33.579803+0000","last_became_active":"2024-09-11T07:57:06.499324+0000","last_became_peered":"2024-09-11T07:57:06.499324+0000","last_unstale":"2024-09-11T07:57:33.579803+0000","last_undegraded":"2024-09-11T07:57:33.579803+0000","last_fullsized":"2024-09-11T07:57:33.579803+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_clean_scrub_stamp":"2024-09-11T07:57:05.472156+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:00:48.941810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.612787+0000","last_change":"2024-09-11T07:57:33.612328+0000","last_active":"2024-09-11T07:57:33.612787+0000","last_peered":"2024-09-11T07:57:33.612787+0000","last_clean":"2024-09-11T07:57:33.612787+0000","last_became_active":"2024-09-11T07:57:33.611979+0000","last_became_peered":"2024-09-11T07:57:33.611979+0000","last_unstale":"2024-09-11T07:57:33.612787+0000","last_undegraded":"2024-09-11T07:57:33.612787+0000","last_fullsized":"2024-09-11T07:57:33.612787+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:31:38.778538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.576936+0000","last_change":"2024-09-11T07:57:08.503430+0000","last_active":"2024-09-11T07:57:33.576936+0000","last_peered":"2024-09-11T07:57:33.576936+0000","last_clean":"2024-09-11T07:57:33.576936+0000","last_became_active":"2024-09-11T07:57:08.503123+0000","last_became_peered":"2024-09-11T07:57:08.503123+0000","last_unstale":"2024-09-11T07:57:33.576936+0000","last_undegraded":"2024-09-11T07:57:33.576936+0000","last_fullsized":"2024-09-11T07:57:33.576936+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_clean_scrub_stamp":"2024-09-11T07:57:07.482151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:02:26.714001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.573057+0000","last_change":"2024-09-11T07:57:10.633968+0000","last_active":"2024-09-11T07:57:33.573057+0000","last_peered":"2024-09-11T07:57:33.573057+0000","last_clean":"2024-09-11T07:57:33.573057+0000","last_became_active":"2024-09-11T07:57:10.633425+0000","last_became_peered":"2024-09-11T07:57:10.633425+0000","last_unstale":"2024-09-11T07:57:33.573057+0000","last_undegraded":"2024-09-11T07:57:33.573057+0000","last_fullsized":"2024-09-11T07:57:33.573057+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_clean_scrub_stamp":"2024-09-11T07:57:09.499431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:32:18.333647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"67'14","reported_seq":39,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.579154+0000","last_change":"2024-09-11T07:57:06.500091+0000","last_active":"2024-09-11T07:57:33.579154+0000","last_peered":"2024-09-11T07:57:33.579154+0000","last_clean":"2024-09-11T07:57:33.579154+0000","last_became_active":"2024-09-11T07:57:06.499802+0000","last_became_peered":"2024-09-11T07:57:06.499802+0000","last_unstale":"2024-09-11T07:57:33.579154+0000","last_undegraded":"2024-09-11T07:57:33.579154+0000","last_fullsized":"2024-09-11T07:57:33.579154+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_clean_scrub_stamp":"2024-09-11T07:57:05.472156+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:46:09.757373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.612262+0000","last_change":"2024-09-11T07:57:04.488374+0000","last_active":"2024-09-11T07:57:33.612262+0000","last_peered":"2024-09-11T07:57:33.612262+0000","last_clean":"2024-09-11T07:57:33.612262+0000","last_became_active":"2024-09-11T07:57:04.488108+0000","last_became_peered":"2024-09-11T07:57:04.488108+0000","last_unstale":"2024-09-11T07:57:33.612262+0000","last_undegraded":"2024-09-11T07:57:33.612262+0000","last_fullsized":"2024-09-11T07:57:33.612262+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:20:32.500248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"67'11","reported_seq":47,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-11T07:58:37.535918+0000","last_change":"2024-09-11T07:57:08.504144+0000","last_active":"2024-09-11T07:58:37.535918+0000","last_peered":"2024-09-11T07:58:37.535918+0000","last_clean":"2024-09-11T07:58:37.535918+0000","last_became_active":"2024-09-11T07:57:08.502566+0000","last_became_peered":"2024-09-11T07:57:08.502566+0000","last_unstale":"2024-09-11T07:58:37.535918+0000","last_undegraded":"2024-09-11T07:58:37.535918+0000","last_fullsized":"2024-09-11T07:58:37.535918+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_clean_scrub_stamp":"2024-09-11T07:57:07.482151+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:26:41.850292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.573477+0000","last_change":"2024-09-11T07:57:10.633883+0000","last_active":"2024-09-11T07:57:33.573477+0000","last_peered":"2024-09-11T07:57:33.573477+0000","last_clean":"2024-09-11T07:57:33.573477+0000","last_became_active":"2024-09-11T07:57:10.633597+0000","last_became_peered":"2024-09-11T07:57:10.633597+0000","last_unstale":"2024-09-11T07:57:33.573477+0000","last_undegraded":"2024-09-11T07:57:33.573477+0000","last_fullsized":"2024-09-11T07:57:33.573477+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_clean_scrub_stamp":"2024-09-11T07:57:09.499431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:27:04.483830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"67'18","reported_seq":50,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.572801+0000","last_change":"2024-09-11T07:57:06.492853+0000","last_active":"2024-09-11T07:57:33.572801+0000","last_peered":"2024-09-11T07:57:33.572801+0000","last_clean":"2024-09-11T07:57:33.572801+0000","last_became_active":"2024-09-11T07:57:06.492556+0000","last_became_peered":"2024-09-11T07:57:06.492556+0000","last_unstale":"2024-09-11T07:57:33.572801+0000","last_undegraded":"2024-09-11T07:57:33.572801+0000","last_fullsized":"2024-09-11T07:57:33.572801+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_clean_scrub_stamp":"2024-09-11T07:57:05.472156+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:08:03.370734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.574370+0000","last_change":"2024-09-11T07:57:04.483733+0000","last_active":"2024-09-11T07:57:33.574370+0000","last_peered":"2024-09-11T07:57:33.574370+0000","last_clean":"2024-09-11T07:57:33.574370+0000","last_became_active":"2024-09-11T07:57:04.483351+0000","last_became_peered":"2024-09-11T07:57:04.483351+0000","last_unstale":"2024-09-11T07:57:33.574370+0000","last_undegraded":"2024-09-11T07:57:33.574370+0000","last_fullsized":"2024-09-11T07:57:33.574370+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:32:09.665437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.579054+0000","last_change":"2024-09-11T07:57:08.505547+0000","last_active":"2024-09-11T07:57:33.579054+0000","last_peered":"2024-09-11T07:57:33.579054+0000","last_clean":"2024-09-11T07:57:33.579054+0000","last_became_active":"2024-09-11T07:57:08.504765+0000","last_became_peered":"2024-09-11T07:57:08.504765+0000","last_unstale":"2024-09-11T07:57:33.579054+0000","last_undegraded":"2024-09-11T07:57:33.579054+0000","last_fullsized":"2024-09-11T07:57:33.579054+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_clean_scrub_stamp":"2024-09-11T07:57:07.482151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:37:32.306648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.579429+0000","last_change":"2024-09-11T07:57:10.516290+0000","last_active":"2024-09-11T07:57:33.579429+0000","last_peered":"2024-09-11T07:57:33.579429+0000","last_clean":"2024-09-11T07:57:33.579429+0000","last_became_active":"2024-09-11T07:57:10.516034+0000","last_became_peered":"2024-09-11T07:57:10.516034+0000","last_unstale":"2024-09-11T07:57:33.579429+0000","last_undegraded":"2024-09-11T07:57:33.579429+0000","last_fullsized":"2024-09-11T07:57:33.579429+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_clean_scrub_stamp":"2024-09-11T07:57:09.499431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:42:38.471181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"67'19","reported_seq":54,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.611699+0000","last_change":"2024-09-11T07:57:06.494748+0000","last_active":"2024-09-11T07:57:33.611699+0000","last_peered":"2024-09-11T07:57:33.611699+0000","last_clean":"2024-09-11T07:57:33.611699+0000","last_became_active":"2024-09-11T07:57:06.493935+0000","last_became_peered":"2024-09-11T07:57:06.493935+0000","last_unstale":"2024-09-11T07:57:33.611699+0000","last_undegraded":"2024-09-11T07:57:33.611699+0000","last_fullsized":"2024-09-11T07:57:33.611699+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_clean_scrub_stamp":"2024-09-11T07:57:05.472156+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:17:29.218282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.573977+0000","last_change":"2024-09-11T07:57:04.491223+0000","last_active":"2024-09-11T07:57:33.573977+0000","last_peered":"2024-09-11T07:57:33.573977+0000","last_clean":"2024-09-11T07:57:33.573977+0000","last_became_active":"2024-09-11T07:57:04.490967+0000","last_became_peered":"2024-09-11T07:57:04.490967+0000","last_unstale":"2024-09-11T07:57:33.573977+0000","last_undegraded":"2024-09-11T07:57:33.573977+0000","last_fullsized":"2024-09-11T07:57:33.573977+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:56:20.746733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.611738+0000","last_change":"2024-09-11T07:57:08.506413+0000","last_active":"2024-09-11T07:57:33.611738+0000","last_peered":"2024-09-11T07:57:33.611738+0000","last_clean":"2024-09-11T07:57:33.611738+0000","last_became_active":"2024-09-11T07:57:08.506108+0000","last_became_peered":"2024-09-11T07:57:08.506108+0000","last_unstale":"2024-09-11T07:57:33.611738+0000","last_undegraded":"2024-09-11T07:57:33.611738+0000","last_fullsized":"2024-09-11T07:57:33.611738+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_clean_scrub_stamp":"2024-09-11T07:57:07.482151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:30:11.388105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.573117+0000","last_change":"2024-09-11T07:57:10.633780+0000","last_active":"2024-09-11T07:57:33.573117+0000","last_peered":"2024-09-11T07:57:33.573117+0000","last_clean":"2024-09-11T07:57:33.573117+0000","last_became_active":"2024-09-11T07:57:10.633413+0000","last_became_peered":"2024-09-11T07:57:10.633413+0000","last_unstale":"2024-09-11T07:57:33.573117+0000","last_undegraded":"2024-09-11T07:57:33.573117+0000","last_fullsized":"2024-09-11T07:57:33.573117+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_clean_scrub_stamp":"2024-09-11T07:57:09.499431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:39:34.058168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"67'10","reported_seq":33,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.574594+0000","last_change":"2024-09-11T07:57:06.496956+0000","last_active":"2024-09-11T07:57:33.574594+0000","last_peered":"2024-09-11T07:57:33.574594+0000","last_clean":"2024-09-11T07:57:33.574594+0000","last_became_active":"2024-09-11T07:57:06.496363+0000","last_became_peered":"2024-09-11T07:57:06.496363+0000","last_unstale":"2024-09-11T07:57:33.574594+0000","last_undegraded":"2024-09-11T07:57:33.574594+0000","last_fullsized":"2024-09-11T07:57:33.574594+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_clean_scrub_stamp":"2024-09-11T07:57:05.472156+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:11:39.417909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.574032+0000","last_change":"2024-09-11T07:57:04.481781+0000","last_active":"2024-09-11T07:57:33.574032+0000","last_peered":"2024-09-11T07:57:33.574032+0000","last_clean":"2024-09-11T07:57:33.574032+0000","last_became_active":"2024-09-11T07:57:04.481523+0000","last_became_peered":"2024-09-11T07:57:04.481523+0000","last_unstale":"2024-09-11T07:57:33.574032+0000","last_undegraded":"2024-09-11T07:57:33.574032+0000","last_fullsized":"2024-09-11T07:57:33.574032+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:01:22.223774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"67'11","reported_seq":47,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-11T07:58:37.535385+0000","last_change":"2024-09-11T07:57:08.506244+0000","last_active":"2024-09-11T07:58:37.535385+0000","last_peered":"2024-09-11T07:58:37.535385+0000","last_clean":"2024-09-11T07:58:37.535385+0000","last_became_active":"2024-09-11T07:57:08.505962+0000","last_became_peered":"2024-09-11T07:57:08.505962+0000","last_unstale":"2024-09-11T07:58:37.535385+0000","last_undegraded":"2024-09-11T07:58:37.535385+0000","last_fullsized":"2024-09-11T07:58:37.535385+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_clean_scrub_stamp":"2024-09-11T07:57:07.482151+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:16:55.639866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.611210+0000","last_change":"2024-09-11T07:57:10.521561+0000","last_active":"2024-09-11T07:57:33.611210+0000","last_peered":"2024-09-11T07:57:33.611210+0000","last_clean":"2024-09-11T07:57:33.611210+0000","last_became_active":"2024-09-11T07:57:10.521023+0000","last_became_peered":"2024-09-11T07:57:10.521023+0000","last_unstale":"2024-09-11T07:57:33.611210+0000","last_undegraded":"2024-09-11T07:57:33.611210+0000","last_fullsized":"2024-09-11T07:57:33.611210+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_clean_scrub_stamp":"2024-09-11T07:57:09.499431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:36:57.173030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"70'31","reported_seq":48,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-11T07:58:37.535548+0000","last_change":"2024-09-11T07:57:33.598370+0000","last_active":"2024-09-11T07:58:37.535548+0000","last_peered":"2024-09-11T07:58:37.535548+0000","last_clean":"2024-09-11T07:58:37.535548+0000","last_became_active":"2024-09-11T07:57:33.580317+0000","last_became_peered":"2024-09-11T07:57:33.580317+0000","last_unstale":"2024-09-11T07:58:37.535548+0000","last_undegraded":"2024-09-11T07:58:37.535548+0000","last_fullsized":"2024-09-11T07:58:37.535548+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_clean_scrub_stamp":"2024-09-11T07:57:05.472156+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:32:56.245828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":51,"num_read_kb":36,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.576462+0000","last_change":"2024-09-11T07:57:04.483428+0000","last_active":"2024-09-11T07:57:33.576462+0000","last_peered":"2024-09-11T07:57:33.576462+0000","last_clean":"2024-09-11T07:57:33.576462+0000","last_became_active":"2024-09-11T07:57:04.482415+0000","last_became_peered":"2024-09-11T07:57:04.482415+0000","last_unstale":"2024-09-11T07:57:33.576462+0000","last_undegraded":"2024-09-11T07:57:33.576462+0000","last_fullsized":"2024-09-11T07:57:33.576462+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:59:07.560339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.611639+0000","last_change":"2024-09-11T07:57:08.508240+0000","last_active":"2024-09-11T07:57:33.611639+0000","last_peered":"2024-09-11T07:57:33.611639+0000","last_clean":"2024-09-11T07:57:33.611639+0000","last_became_active":"2024-09-11T07:57:08.507927+0000","last_became_peered":"2024-09-11T07:57:08.507927+0000","last_unstale":"2024-09-11T07:57:33.611639+0000","last_undegraded":"2024-09-11T07:57:33.611639+0000","last_fullsized":"2024-09-11T07:57:33.611639+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_clean_scrub_stamp":"2024-09-11T07:57:07.482151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:43:56.913115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.573691+0000","last_change":"2024-09-11T07:57:10.517053+0000","last_active":"2024-09-11T07:57:33.573691+0000","last_peered":"2024-09-11T07:57:33.573691+0000","last_clean":"2024-09-11T07:57:33.573691+0000","last_became_active":"2024-09-11T07:57:10.516762+0000","last_became_peered":"2024-09-11T07:57:10.516762+0000","last_unstale":"2024-09-11T07:57:33.573691+0000","last_undegraded":"2024-09-11T07:57:33.573691+0000","last_fullsized":"2024-09-11T07:57:33.573691+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_clean_scrub_stamp":"2024-09-11T07:57:09.499431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:32:50.888196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"67'13","reported_seq":45,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.574101+0000","last_change":"2024-09-11T07:57:06.492607+0000","last_active":"2024-09-11T07:57:33.574101+0000","last_peered":"2024-09-11T07:57:33.574101+0000","last_clean":"2024-09-11T07:57:33.574101+0000","last_became_active":"2024-09-11T07:57:06.492341+0000","last_became_peered":"2024-09-11T07:57:06.492341+0000","last_unstale":"2024-09-11T07:57:33.574101+0000","last_undegraded":"2024-09-11T07:57:33.574101+0000","last_fullsized":"2024-09-11T07:57:33.574101+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_clean_scrub_stamp":"2024-09-11T07:57:05.472156+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:30:00.910172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"67'5","reported_seq":120,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-11T07:58:41.446667+0000","last_change":"2024-09-11T07:57:09.507219+0000","last_active":"2024-09-11T07:58:41.446667+0000","last_peered":"2024-09-11T07:58:41.446667+0000","last_clean":"2024-09-11T07:58:41.446667+0000","last_became_active":"2024-09-11T07:57:04.485435+0000","last_became_peered":"2024-09-11T07:57:04.485435+0000","last_unstale":"2024-09-11T07:58:41.446667+0000","last_undegraded":"2024-09-11T07:58:41.446667+0000","last_fullsized":"2024-09-11T07:58:41.446667+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:23.345662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011489740000000001,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":92,"num_read_kb":87,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.576238+0000","last_change":"2024-09-11T07:57:04.481079+0000","last_active":"2024-09-11T07:57:33.576238+0000","last_peered":"2024-09-11T07:57:33.576238+0000","last_clean":"2024-09-11T07:57:33.576238+0000","last_became_active":"2024-09-11T07:57:04.480774+0000","last_became_peered":"2024-09-11T07:57:04.480774+0000","last_unstale":"2024-09-11T07:57:33.576238+0000","last_undegraded":"2024-09-11T07:57:33.576238+0000","last_fullsized":"2024-09-11T07:57:33.576238+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:55:43.762836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.577379+0000","last_change":"2024-09-11T07:57:08.504850+0000","last_active":"2024-09-11T07:57:33.577379+0000","last_peered":"2024-09-11T07:57:33.577379+0000","last_clean":"2024-09-11T07:57:33.577379+0000","last_became_active":"2024-09-11T07:57:08.504465+0000","last_became_peered":"2024-09-11T07:57:08.504465+0000","last_unstale":"2024-09-11T07:57:33.577379+0000","last_undegraded":"2024-09-11T07:57:33.577379+0000","last_fullsized":"2024-09-11T07:57:33.577379+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_clean_scrub_stamp":"2024-09-11T07:57:07.482151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:06:42.204281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"67'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.573615+0000","last_change":"2024-09-11T07:57:10.635491+0000","last_active":"2024-09-11T07:57:33.573615+0000","last_peered":"2024-09-11T07:57:33.573615+0000","last_clean":"2024-09-11T07:57:33.573615+0000","last_became_active":"2024-09-11T07:57:10.635197+0000","last_became_peered":"2024-09-11T07:57:10.635197+0000","last_unstale":"2024-09-11T07:57:33.573615+0000","last_undegraded":"2024-09-11T07:57:33.573615+0000","last_fullsized":"2024-09-11T07:57:33.573615+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_clean_scrub_stamp":"2024-09-11T07:57:09.499431+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:07:50.407303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"67'12","reported_seq":36,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.611964+0000","last_change":"2024-09-11T07:57:06.495472+0000","last_active":"2024-09-11T07:57:33.611964+0000","last_peered":"2024-09-11T07:57:33.611964+0000","last_clean":"2024-09-11T07:57:33.611964+0000","last_became_active":"2024-09-11T07:57:06.495225+0000","last_became_peered":"2024-09-11T07:57:06.495225+0000","last_unstale":"2024-09-11T07:57:33.611964+0000","last_undegraded":"2024-09-11T07:57:33.611964+0000","last_fullsized":"2024-09-11T07:57:33.611964+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_clean_scrub_stamp":"2024-09-11T07:57:05.472156+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:13:28.038198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"63'1","reported_seq":26,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.611334+0000","last_change":"2024-09-11T07:57:09.506152+0000","last_active":"2024-09-11T07:57:33.611334+0000","last_peered":"2024-09-11T07:57:33.611334+0000","last_clean":"2024-09-11T07:57:33.611334+0000","last_became_active":"2024-09-11T07:57:04.483726+0000","last_became_peered":"2024-09-11T07:57:04.483726+0000","last_unstale":"2024-09-11T07:57:33.611334+0000","last_undegraded":"2024-09-11T07:57:33.611334+0000","last_fullsized":"2024-09-11T07:57:33.611334+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:30:02.189132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011813679999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.573455+0000","last_change":"2024-09-11T07:57:04.492972+0000","last_active":"2024-09-11T07:57:33.573455+0000","last_peered":"2024-09-11T07:57:33.573455+0000","last_clean":"2024-09-11T07:57:33.573455+0000","last_became_active":"2024-09-11T07:57:04.492611+0000","last_became_peered":"2024-09-11T07:57:04.492611+0000","last_unstale":"2024-09-11T07:57:33.573455+0000","last_undegraded":"2024-09-11T07:57:33.573455+0000","last_fullsized":"2024-09-11T07:57:33.573455+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:57:56.550858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.577027+0000","last_change":"2024-09-11T07:57:08.511826+0000","last_active":"2024-09-11T07:57:33.577027+0000","last_peered":"2024-09-11T07:57:33.577027+0000","last_clean":"2024-09-11T07:57:33.577027+0000","last_became_active":"2024-09-11T07:57:08.511505+0000","last_became_peered":"2024-09-11T07:57:08.511505+0000","last_unstale":"2024-09-11T07:57:33.577027+0000","last_undegraded":"2024-09-11T07:57:33.577027+0000","last_fullsized":"2024-09-11T07:57:33.577027+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_clean_scrub_stamp":"2024-09-11T07:57:07.482151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:38:51.881140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.573502+0000","last_change":"2024-09-11T07:57:10.514505+0000","last_active":"2024-09-11T07:57:33.573502+0000","last_peered":"2024-09-11T07:57:33.573502+0000","last_clean":"2024-09-11T07:57:33.573502+0000","last_became_active":"2024-09-11T07:57:10.514044+0000","last_became_peered":"2024-09-11T07:57:10.514044+0000","last_unstale":"2024-09-11T07:57:33.573502+0000","last_undegraded":"2024-09-11T07:57:33.573502+0000","last_fullsized":"2024-09-11T07:57:33.573502+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_clean_scrub_stamp":"2024-09-11T07:57:09.499431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:32:51.813243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"67'14","reported_seq":40,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.612052+0000","last_change":"2024-09-11T07:57:06.497135+0000","last_active":"2024-09-11T07:57:33.612052+0000","last_peered":"2024-09-11T07:57:33.612052+0000","last_clean":"2024-09-11T07:57:33.612052+0000","last_became_active":"2024-09-11T07:57:06.496742+0000","last_became_peered":"2024-09-11T07:57:06.496742+0000","last_unstale":"2024-09-11T07:57:33.612052+0000","last_undegraded":"2024-09-11T07:57:33.612052+0000","last_fullsized":"2024-09-11T07:57:33.612052+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_clean_scrub_stamp":"2024-09-11T07:57:05.472156+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:02.966730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"65'2","reported_seq":27,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.573072+0000","last_change":"2024-09-11T07:57:09.511590+0000","last_active":"2024-09-11T07:57:33.573072+0000","last_peered":"2024-09-11T07:57:33.573072+0000","last_clean":"2024-09-11T07:57:33.573072+0000","last_became_active":"2024-09-11T07:57:04.483268+0000","last_became_peered":"2024-09-11T07:57:04.483268+0000","last_unstale":"2024-09-11T07:57:33.573072+0000","last_undegraded":"2024-09-11T07:57:33.573072+0000","last_fullsized":"2024-09-11T07:57:33.573072+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:39:36.156196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010013400000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.576607+0000","last_change":"2024-09-11T07:57:04.489072+0000","last_active":"2024-09-11T07:57:33.576607+0000","last_peered":"2024-09-11T07:57:33.576607+0000","last_clean":"2024-09-11T07:57:33.576607+0000","last_became_active":"2024-09-11T07:57:04.488800+0000","last_became_peered":"2024-09-11T07:57:04.488800+0000","last_unstale":"2024-09-11T07:57:33.576607+0000","last_undegraded":"2024-09-11T07:57:33.576607+0000","last_fullsized":"2024-09-11T07:57:33.576607+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:03:10.848604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"71'139","reported_seq":273,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-11T07:57:49.027960+0000","last_change":"2024-09-11T07:56:54.440968+0000","last_active":"2024-09-11T07:57:49.027960+0000","last_peered":"2024-09-11T07:57:49.027960+0000","last_clean":"2024-09-11T07:57:49.027960+0000","last_became_active":"2024-09-11T07:56:54.440625+0000","last_became_peered":"2024-09-11T07:56:54.440625+0000","last_unstale":"2024-09-11T07:57:49.027960+0000","last_undegraded":"2024-09-11T07:57:49.027960+0000","last_fullsized":"2024-09-11T07:57:49.027960+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:54:50.621078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:54:50.621078+0000","last_clean_scrub_stamp":"2024-09-11T07:54:50.621078+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:19:00.916869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":156,"num_read_kb":258,"num_write":179,"num_write_kb":3194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.573449+0000","last_change":"2024-09-11T07:57:08.508684+0000","last_active":"2024-09-11T07:57:33.573449+0000","last_peered":"2024-09-11T07:57:33.573449+0000","last_clean":"2024-09-11T07:57:33.573449+0000","last_became_active":"2024-09-11T07:57:08.508410+0000","last_became_peered":"2024-09-11T07:57:08.508410+0000","last_unstale":"2024-09-11T07:57:33.573449+0000","last_undegraded":"2024-09-11T07:57:33.573449+0000","last_fullsized":"2024-09-11T07:57:33.573449+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_clean_scrub_stamp":"2024-09-11T07:57:07.482151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:08.594058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.577051+0000","last_change":"2024-09-11T07:57:10.515110+0000","last_active":"2024-09-11T07:57:33.577051+0000","last_peered":"2024-09-11T07:57:33.577051+0000","last_clean":"2024-09-11T07:57:33.577051+0000","last_became_active":"2024-09-11T07:57:10.514819+0000","last_became_peered":"2024-09-11T07:57:10.514819+0000","last_unstale":"2024-09-11T07:57:33.577051+0000","last_undegraded":"2024-09-11T07:57:33.577051+0000","last_fullsized":"2024-09-11T07:57:33.577051+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_clean_scrub_stamp":"2024-09-11T07:57:09.499431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:59:12.065651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"67'19","reported_seq":49,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.612140+0000","last_change":"2024-09-11T07:57:06.500416+0000","last_active":"2024-09-11T07:57:33.612140+0000","last_peered":"2024-09-11T07:57:33.612140+0000","last_clean":"2024-09-11T07:57:33.612140+0000","last_became_active":"2024-09-11T07:57:06.500023+0000","last_became_peered":"2024-09-11T07:57:06.500023+0000","last_unstale":"2024-09-11T07:57:33.612140+0000","last_undegraded":"2024-09-11T07:57:33.612140+0000","last_fullsized":"2024-09-11T07:57:33.612140+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_clean_scrub_stamp":"2024-09-11T07:57:05.472156+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:36:13.697483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.578870+0000","last_change":"2024-09-11T07:57:04.484400+0000","last_active":"2024-09-11T07:57:33.578870+0000","last_peered":"2024-09-11T07:57:33.578870+0000","last_clean":"2024-09-11T07:57:33.578870+0000","last_became_active":"2024-09-11T07:57:04.484081+0000","last_became_peered":"2024-09-11T07:57:04.484081+0000","last_unstale":"2024-09-11T07:57:33.578870+0000","last_undegraded":"2024-09-11T07:57:33.578870+0000","last_fullsized":"2024-09-11T07:57:33.578870+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:46:59.929296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.577025+0000","last_change":"2024-09-11T07:57:08.506946+0000","last_active":"2024-09-11T07:57:33.577025+0000","last_peered":"2024-09-11T07:57:33.577025+0000","last_clean":"2024-09-11T07:57:33.577025+0000","last_became_active":"2024-09-11T07:57:08.506646+0000","last_became_peered":"2024-09-11T07:57:08.506646+0000","last_unstale":"2024-09-11T07:57:33.577025+0000","last_undegraded":"2024-09-11T07:57:33.577025+0000","last_fullsized":"2024-09-11T07:57:33.577025+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_clean_scrub_stamp":"2024-09-11T07:57:07.482151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:42:42.884334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.573570+0000","last_change":"2024-09-11T07:57:10.521840+0000","last_active":"2024-09-11T07:57:33.573570+0000","last_peered":"2024-09-11T07:57:33.573570+0000","last_clean":"2024-09-11T07:57:33.573570+0000","last_became_active":"2024-09-11T07:57:10.521574+0000","last_became_peered":"2024-09-11T07:57:10.521574+0000","last_unstale":"2024-09-11T07:57:33.573570+0000","last_undegraded":"2024-09-11T07:57:33.573570+0000","last_fullsized":"2024-09-11T07:57:33.573570+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_clean_scrub_stamp":"2024-09-11T07:57:09.499431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:58:43.959712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"67'9","reported_seq":34,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.611586+0000","last_change":"2024-09-11T07:57:06.496305+0000","last_active":"2024-09-11T07:57:33.611586+0000","last_peered":"2024-09-11T07:57:33.611586+0000","last_clean":"2024-09-11T07:57:33.611586+0000","last_became_active":"2024-09-11T07:57:06.495261+0000","last_became_peered":"2024-09-11T07:57:06.495261+0000","last_unstale":"2024-09-11T07:57:33.611586+0000","last_undegraded":"2024-09-11T07:57:33.611586+0000","last_fullsized":"2024-09-11T07:57:33.611586+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_clean_scrub_stamp":"2024-09-11T07:57:05.472156+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:27:24.348733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.574395+0000","last_change":"2024-09-11T07:57:04.484471+0000","last_active":"2024-09-11T07:57:33.574395+0000","last_peered":"2024-09-11T07:57:33.574395+0000","last_clean":"2024-09-11T07:57:33.574395+0000","last_became_active":"2024-09-11T07:57:04.484170+0000","last_became_peered":"2024-09-11T07:57:04.484170+0000","last_unstale":"2024-09-11T07:57:33.574395+0000","last_undegraded":"2024-09-11T07:57:33.574395+0000","last_fullsized":"2024-09-11T07:57:33.574395+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:12:07.946781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.576557+0000","last_change":"2024-09-11T07:57:08.505698+0000","last_active":"2024-09-11T07:57:33.576557+0000","last_peered":"2024-09-11T07:57:33.576557+0000","last_clean":"2024-09-11T07:57:33.576557+0000","last_became_active":"2024-09-11T07:57:08.505391+0000","last_became_peered":"2024-09-11T07:57:08.505391+0000","last_unstale":"2024-09-11T07:57:33.576557+0000","last_undegraded":"2024-09-11T07:57:33.576557+0000","last_fullsized":"2024-09-11T07:57:33.576557+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_clean_scrub_stamp":"2024-09-11T07:57:07.482151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:12:19.857864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.611282+0000","last_change":"2024-09-11T07:57:10.521482+0000","last_active":"2024-09-11T07:57:33.611282+0000","last_peered":"2024-09-11T07:57:33.611282+0000","last_clean":"2024-09-11T07:57:33.611282+0000","last_became_active":"2024-09-11T07:57:10.521053+0000","last_became_peered":"2024-09-11T07:57:10.521053+0000","last_unstale":"2024-09-11T07:57:33.611282+0000","last_undegraded":"2024-09-11T07:57:33.611282+0000","last_fullsized":"2024-09-11T07:57:33.611282+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_clean_scrub_stamp":"2024-09-11T07:57:09.499431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:15:54.897151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"67'15","reported_seq":43,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.575997+0000","last_change":"2024-09-11T07:57:06.494611+0000","last_active":"2024-09-11T07:57:33.575997+0000","last_peered":"2024-09-11T07:57:33.575997+0000","last_clean":"2024-09-11T07:57:33.575997+0000","last_became_active":"2024-09-11T07:57:06.494356+0000","last_became_peered":"2024-09-11T07:57:06.494356+0000","last_unstale":"2024-09-11T07:57:33.575997+0000","last_undegraded":"2024-09-11T07:57:33.575997+0000","last_fullsized":"2024-09-11T07:57:33.575997+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_clean_scrub_stamp":"2024-09-11T07:57:05.472156+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:45:26.230617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.578587+0000","last_change":"2024-09-11T07:57:04.487620+0000","last_active":"2024-09-11T07:57:33.578587+0000","last_peered":"2024-09-11T07:57:33.578587+0000","last_clean":"2024-09-11T07:57:33.578587+0000","last_became_active":"2024-09-11T07:57:04.487356+0000","last_became_peered":"2024-09-11T07:57:04.487356+0000","last_unstale":"2024-09-11T07:57:33.578587+0000","last_undegraded":"2024-09-11T07:57:33.578587+0000","last_fullsized":"2024-09-11T07:57:33.578587+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:44:19.856124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"67'11","reported_seq":47,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-11T07:58:37.535330+0000","last_change":"2024-09-11T07:57:08.512715+0000","last_active":"2024-09-11T07:58:37.535330+0000","last_peered":"2024-09-11T07:58:37.535330+0000","last_clean":"2024-09-11T07:58:37.535330+0000","last_became_active":"2024-09-11T07:57:08.510784+0000","last_became_peered":"2024-09-11T07:57:08.510784+0000","last_unstale":"2024-09-11T07:58:37.535330+0000","last_undegraded":"2024-09-11T07:58:37.535330+0000","last_fullsized":"2024-09-11T07:58:37.535330+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_clean_scrub_stamp":"2024-09-11T07:57:07.482151+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:24:03.571661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.576151+0000","last_change":"2024-09-11T07:57:10.633731+0000","last_active":"2024-09-11T07:57:33.576151+0000","last_peered":"2024-09-11T07:57:33.576151+0000","last_clean":"2024-09-11T07:57:33.576151+0000","last_became_active":"2024-09-11T07:57:10.633431+0000","last_became_peered":"2024-09-11T07:57:10.633431+0000","last_unstale":"2024-09-11T07:57:33.576151+0000","last_undegraded":"2024-09-11T07:57:33.576151+0000","last_fullsized":"2024-09-11T07:57:33.576151+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_clean_scrub_stamp":"2024-09-11T07:57:09.499431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:45:58.755051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"67'12","reported_seq":41,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.579039+0000","last_change":"2024-09-11T07:57:06.498941+0000","last_active":"2024-09-11T07:57:33.579039+0000","last_peered":"2024-09-11T07:57:33.579039+0000","last_clean":"2024-09-11T07:57:33.579039+0000","last_became_active":"2024-09-11T07:57:06.498679+0000","last_became_peered":"2024-09-11T07:57:06.498679+0000","last_unstale":"2024-09-11T07:57:33.579039+0000","last_undegraded":"2024-09-11T07:57:33.579039+0000","last_fullsized":"2024-09-11T07:57:33.579039+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_clean_scrub_stamp":"2024-09-11T07:57:05.472156+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:27:32.243007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"60'2","reported_seq":42,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.580336+0000","last_change":"2024-09-11T07:57:04.484156+0000","last_active":"2024-09-11T07:57:33.580336+0000","last_peered":"2024-09-11T07:57:33.580336+0000","last_clean":"2024-09-11T07:57:33.580336+0000","last_became_active":"2024-09-11T07:57:04.483858+0000","last_became_peered":"2024-09-11T07:57:04.483858+0000","last_unstale":"2024-09-11T07:57:33.580336+0000","last_undegraded":"2024-09-11T07:57:33.580336+0000","last_fullsized":"2024-09-11T07:57:33.580336+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:57:33.680551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.577179+0000","last_change":"2024-09-11T07:57:08.508100+0000","last_active":"2024-09-11T07:57:33.577179+0000","last_peered":"2024-09-11T07:57:33.577179+0000","last_clean":"2024-09-11T07:57:33.577179+0000","last_became_active":"2024-09-11T07:57:08.507823+0000","last_became_peered":"2024-09-11T07:57:08.507823+0000","last_unstale":"2024-09-11T07:57:33.577179+0000","last_undegraded":"2024-09-11T07:57:33.577179+0000","last_fullsized":"2024-09-11T07:57:33.577179+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_clean_scrub_stamp":"2024-09-11T07:57:07.482151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:01:32.478843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.572900+0000","last_change":"2024-09-11T07:57:10.517901+0000","last_active":"2024-09-11T07:57:33.572900+0000","last_peered":"2024-09-11T07:57:33.572900+0000","last_clean":"2024-09-11T07:57:33.572900+0000","last_became_active":"2024-09-11T07:57:10.517630+0000","last_became_peered":"2024-09-11T07:57:10.517630+0000","last_unstale":"2024-09-11T07:57:33.572900+0000","last_undegraded":"2024-09-11T07:57:33.572900+0000","last_fullsized":"2024-09-11T07:57:33.572900+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_clean_scrub_stamp":"2024-09-11T07:57:09.499431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:24:49.152386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"67'9","reported_seq":34,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.611955+0000","last_change":"2024-09-11T07:57:06.494360+0000","last_active":"2024-09-11T07:57:33.611955+0000","last_peered":"2024-09-11T07:57:33.611955+0000","last_clean":"2024-09-11T07:57:33.611955+0000","last_became_active":"2024-09-11T07:57:06.494097+0000","last_became_peered":"2024-09-11T07:57:06.494097+0000","last_unstale":"2024-09-11T07:57:33.611955+0000","last_undegraded":"2024-09-11T07:57:33.611955+0000","last_fullsized":"2024-09-11T07:57:33.611955+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_clean_scrub_stamp":"2024-09-11T07:57:05.472156+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:53:27.619145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"60'1","reported_seq":31,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.577466+0000","last_change":"2024-09-11T07:57:04.481567+0000","last_active":"2024-09-11T07:57:33.577466+0000","last_peered":"2024-09-11T07:57:33.577466+0000","last_clean":"2024-09-11T07:57:33.577466+0000","last_became_active":"2024-09-11T07:57:04.481303+0000","last_became_peered":"2024-09-11T07:57:04.481303+0000","last_unstale":"2024-09-11T07:57:33.577466+0000","last_undegraded":"2024-09-11T07:57:33.577466+0000","last_fullsized":"2024-09-11T07:57:33.577466+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:03:21.719486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.574635+0000","last_change":"2024-09-11T07:57:08.502302+0000","last_active":"2024-09-11T07:57:33.574635+0000","last_peered":"2024-09-11T07:57:33.574635+0000","last_clean":"2024-09-11T07:57:33.574635+0000","last_became_active":"2024-09-11T07:57:08.502033+0000","last_became_peered":"2024-09-11T07:57:08.502033+0000","last_unstale":"2024-09-11T07:57:33.574635+0000","last_undegraded":"2024-09-11T07:57:33.574635+0000","last_fullsized":"2024-09-11T07:57:33.574635+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_clean_scrub_stamp":"2024-09-11T07:57:07.482151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:16:39.551182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.578504+0000","last_change":"2024-09-11T07:57:10.519872+0000","last_active":"2024-09-11T07:57:33.578504+0000","last_peered":"2024-09-11T07:57:33.578504+0000","last_clean":"2024-09-11T07:57:33.578504+0000","last_became_active":"2024-09-11T07:57:10.519607+0000","last_became_peered":"2024-09-11T07:57:10.519607+0000","last_unstale":"2024-09-11T07:57:33.578504+0000","last_undegraded":"2024-09-11T07:57:33.578504+0000","last_fullsized":"2024-09-11T07:57:33.578504+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_clean_scrub_stamp":"2024-09-11T07:57:09.499431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:26:12.890135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.611880+0000","last_change":"2024-09-11T07:57:04.487573+0000","last_active":"2024-09-11T07:57:33.611880+0000","last_peered":"2024-09-11T07:57:33.611880+0000","last_clean":"2024-09-11T07:57:33.611880+0000","last_became_active":"2024-09-11T07:57:04.487303+0000","last_became_peered":"2024-09-11T07:57:04.487303+0000","last_unstale":"2024-09-11T07:57:33.611880+0000","last_undegraded":"2024-09-11T07:57:33.611880+0000","last_fullsized":"2024-09-11T07:57:33.611880+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:14:29.875410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"67'6","reported_seq":27,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.574819+0000","last_change":"2024-09-11T07:57:06.487817+0000","last_active":"2024-09-11T07:57:33.574819+0000","last_peered":"2024-09-11T07:57:33.574819+0000","last_clean":"2024-09-11T07:57:33.574819+0000","last_became_active":"2024-09-11T07:57:06.487555+0000","last_became_peered":"2024-09-11T07:57:06.487555+0000","last_unstale":"2024-09-11T07:57:33.574819+0000","last_undegraded":"2024-09-11T07:57:33.574819+0000","last_fullsized":"2024-09-11T07:57:33.574819+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_clean_scrub_stamp":"2024-09-11T07:57:05.472156+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:17:18.051967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.576824+0000","last_change":"2024-09-11T07:57:08.508247+0000","last_active":"2024-09-11T07:57:33.576824+0000","last_peered":"2024-09-11T07:57:33.576824+0000","last_clean":"2024-09-11T07:57:33.576824+0000","last_became_active":"2024-09-11T07:57:08.507947+0000","last_became_peered":"2024-09-11T07:57:08.507947+0000","last_unstale":"2024-09-11T07:57:33.576824+0000","last_undegraded":"2024-09-11T07:57:33.576824+0000","last_fullsized":"2024-09-11T07:57:33.576824+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_clean_scrub_stamp":"2024-09-11T07:57:07.482151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:14:37.773015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.573385+0000","last_change":"2024-09-11T07:57:10.635467+0000","last_active":"2024-09-11T07:57:33.573385+0000","last_peered":"2024-09-11T07:57:33.573385+0000","last_clean":"2024-09-11T07:57:33.573385+0000","last_became_active":"2024-09-11T07:57:10.635177+0000","last_became_peered":"2024-09-11T07:57:10.635177+0000","last_unstale":"2024-09-11T07:57:33.573385+0000","last_undegraded":"2024-09-11T07:57:33.573385+0000","last_fullsized":"2024-09-11T07:57:33.573385+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_clean_scrub_stamp":"2024-09-11T07:57:09.499431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:09:31.255711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"67'10","reported_seq":33,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.573857+0000","last_change":"2024-09-11T07:57:06.488456+0000","last_active":"2024-09-11T07:57:33.573857+0000","last_peered":"2024-09-11T07:57:33.573857+0000","last_clean":"2024-09-11T07:57:33.573857+0000","last_became_active":"2024-09-11T07:57:06.488206+0000","last_became_peered":"2024-09-11T07:57:06.488206+0000","last_unstale":"2024-09-11T07:57:33.573857+0000","last_undegraded":"2024-09-11T07:57:33.573857+0000","last_fullsized":"2024-09-11T07:57:33.573857+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_clean_scrub_stamp":"2024-09-11T07:57:05.472156+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:36:00.116993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.576537+0000","last_change":"2024-09-11T07:57:04.487787+0000","last_active":"2024-09-11T07:57:33.576537+0000","last_peered":"2024-09-11T07:57:33.576537+0000","last_clean":"2024-09-11T07:57:33.576537+0000","last_became_active":"2024-09-11T07:57:04.487317+0000","last_became_peered":"2024-09-11T07:57:04.487317+0000","last_unstale":"2024-09-11T07:57:33.576537+0000","last_undegraded":"2024-09-11T07:57:33.576537+0000","last_fullsized":"2024-09-11T07:57:33.576537+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:36:55.078580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"67'11","reported_seq":50,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-11T07:58:37.535359+0000","last_change":"2024-09-11T07:57:08.509403+0000","last_active":"2024-09-11T07:58:37.535359+0000","last_peered":"2024-09-11T07:58:37.535359+0000","last_clean":"2024-09-11T07:58:37.535359+0000","last_became_active":"2024-09-11T07:57:08.508994+0000","last_became_peered":"2024-09-11T07:57:08.508994+0000","last_unstale":"2024-09-11T07:58:37.535359+0000","last_undegraded":"2024-09-11T07:58:37.535359+0000","last_fullsized":"2024-09-11T07:58:37.535359+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_clean_scrub_stamp":"2024-09-11T07:57:07.482151+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:43:29.753815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.611546+0000","last_change":"2024-09-11T07:57:10.519763+0000","last_active":"2024-09-11T07:57:33.611546+0000","last_peered":"2024-09-11T07:57:33.611546+0000","last_clean":"2024-09-11T07:57:33.611546+0000","last_became_active":"2024-09-11T07:57:10.519464+0000","last_became_peered":"2024-09-11T07:57:10.519464+0000","last_unstale":"2024-09-11T07:57:33.611546+0000","last_undegraded":"2024-09-11T07:57:33.611546+0000","last_fullsized":"2024-09-11T07:57:33.611546+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_clean_scrub_stamp":"2024-09-11T07:57:09.499431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:10:12.376281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"67'9","reported_seq":34,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.576724+0000","last_change":"2024-09-11T07:57:06.492782+0000","last_active":"2024-09-11T07:57:33.576724+0000","last_peered":"2024-09-11T07:57:33.576724+0000","last_clean":"2024-09-11T07:57:33.576724+0000","last_became_active":"2024-09-11T07:57:06.492521+0000","last_became_peered":"2024-09-11T07:57:06.492521+0000","last_unstale":"2024-09-11T07:57:33.576724+0000","last_undegraded":"2024-09-11T07:57:33.576724+0000","last_fullsized":"2024-09-11T07:57:33.576724+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_clean_scrub_stamp":"2024-09-11T07:57:05.472156+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:59:33.293124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.611847+0000","last_change":"2024-09-11T07:57:04.484744+0000","last_active":"2024-09-11T07:57:33.611847+0000","last_peered":"2024-09-11T07:57:33.611847+0000","last_clean":"2024-09-11T07:57:33.611847+0000","last_became_active":"2024-09-11T07:57:04.483807+0000","last_became_peered":"2024-09-11T07:57:04.483807+0000","last_unstale":"2024-09-11T07:57:33.611847+0000","last_undegraded":"2024-09-11T07:57:33.611847+0000","last_fullsized":"2024-09-11T07:57:33.611847+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:12:27.228249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"67'11","reported_seq":47,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-11T07:58:37.535921+0000","last_change":"2024-09-11T07:57:08.500036+0000","last_active":"2024-09-11T07:58:37.535921+0000","last_peered":"2024-09-11T07:58:37.535921+0000","last_clean":"2024-09-11T07:58:37.535921+0000","last_became_active":"2024-09-11T07:57:08.499732+0000","last_became_peered":"2024-09-11T07:57:08.499732+0000","last_unstale":"2024-09-11T07:58:37.535921+0000","last_undegraded":"2024-09-11T07:58:37.535921+0000","last_fullsized":"2024-09-11T07:58:37.535921+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_clean_scrub_stamp":"2024-09-11T07:57:07.482151+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:04:59.423555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.580100+0000","last_change":"2024-09-11T07:57:10.513630+0000","last_active":"2024-09-11T07:57:33.580100+0000","last_peered":"2024-09-11T07:57:33.580100+0000","last_clean":"2024-09-11T07:57:33.580100+0000","last_became_active":"2024-09-11T07:57:10.513250+0000","last_became_peered":"2024-09-11T07:57:10.513250+0000","last_unstale":"2024-09-11T07:57:33.580100+0000","last_undegraded":"2024-09-11T07:57:33.580100+0000","last_fullsized":"2024-09-11T07:57:33.580100+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_clean_scrub_stamp":"2024-09-11T07:57:09.499431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:06:46.311189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"67'9","reported_seq":34,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.574449+0000","last_change":"2024-09-11T07:57:06.495694+0000","last_active":"2024-09-11T07:57:33.574449+0000","last_peered":"2024-09-11T07:57:33.574449+0000","last_clean":"2024-09-11T07:57:33.574449+0000","last_became_active":"2024-09-11T07:57:06.495434+0000","last_became_peered":"2024-09-11T07:57:06.495434+0000","last_unstale":"2024-09-11T07:57:33.574449+0000","last_undegraded":"2024-09-11T07:57:33.574449+0000","last_fullsized":"2024-09-11T07:57:33.574449+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_clean_scrub_stamp":"2024-09-11T07:57:05.472156+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:14:21.915281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.612110+0000","last_change":"2024-09-11T07:57:04.485350+0000","last_active":"2024-09-11T07:57:33.612110+0000","last_peered":"2024-09-11T07:57:33.612110+0000","last_clean":"2024-09-11T07:57:33.612110+0000","last_became_active":"2024-09-11T07:57:04.485096+0000","last_became_peered":"2024-09-11T07:57:04.485096+0000","last_unstale":"2024-09-11T07:57:33.612110+0000","last_undegraded":"2024-09-11T07:57:33.612110+0000","last_fullsized":"2024-09-11T07:57:33.612110+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:26:19.478481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.572989+0000","last_change":"2024-09-11T07:57:08.501971+0000","last_active":"2024-09-11T07:57:33.572989+0000","last_peered":"2024-09-11T07:57:33.572989+0000","last_clean":"2024-09-11T07:57:33.572989+0000","last_became_active":"2024-09-11T07:57:08.501689+0000","last_became_peered":"2024-09-11T07:57:08.501689+0000","last_unstale":"2024-09-11T07:57:33.572989+0000","last_undegraded":"2024-09-11T07:57:33.572989+0000","last_fullsized":"2024-09-11T07:57:33.572989+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_clean_scrub_stamp":"2024-09-11T07:57:07.482151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:52:21.706527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.611110+0000","last_change":"2024-09-11T07:57:10.515638+0000","last_active":"2024-09-11T07:57:33.611110+0000","last_peered":"2024-09-11T07:57:33.611110+0000","last_clean":"2024-09-11T07:57:33.611110+0000","last_became_active":"2024-09-11T07:57:10.515363+0000","last_became_peered":"2024-09-11T07:57:10.515363+0000","last_unstale":"2024-09-11T07:57:33.611110+0000","last_undegraded":"2024-09-11T07:57:33.611110+0000","last_fullsized":"2024-09-11T07:57:33.611110+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_clean_scrub_stamp":"2024-09-11T07:57:09.499431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:37:06.037292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"67'11","reported_seq":37,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.579479+0000","last_change":"2024-09-11T07:57:06.498266+0000","last_active":"2024-09-11T07:57:33.579479+0000","last_peered":"2024-09-11T07:57:33.579479+0000","last_clean":"2024-09-11T07:57:33.579479+0000","last_became_active":"2024-09-11T07:57:06.497988+0000","last_became_peered":"2024-09-11T07:57:06.497988+0000","last_unstale":"2024-09-11T07:57:33.579479+0000","last_undegraded":"2024-09-11T07:57:33.579479+0000","last_fullsized":"2024-09-11T07:57:33.579479+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_clean_scrub_stamp":"2024-09-11T07:57:05.472156+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:48:58.577105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.574310+0000","last_change":"2024-09-11T07:57:04.480914+0000","last_active":"2024-09-11T07:57:33.574310+0000","last_peered":"2024-09-11T07:57:33.574310+0000","last_clean":"2024-09-11T07:57:33.574310+0000","last_became_active":"2024-09-11T07:57:04.480635+0000","last_became_peered":"2024-09-11T07:57:04.480635+0000","last_unstale":"2024-09-11T07:57:33.574310+0000","last_undegraded":"2024-09-11T07:57:33.574310+0000","last_fullsized":"2024-09-11T07:57:33.574310+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:59:41.799592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.573374+0000","last_change":"2024-09-11T07:57:08.505170+0000","last_active":"2024-09-11T07:57:33.573374+0000","last_peered":"2024-09-11T07:57:33.573374+0000","last_clean":"2024-09-11T07:57:33.573374+0000","last_became_active":"2024-09-11T07:57:08.504855+0000","last_became_peered":"2024-09-11T07:57:08.504855+0000","last_unstale":"2024-09-11T07:57:33.573374+0000","last_undegraded":"2024-09-11T07:57:33.573374+0000","last_fullsized":"2024-09-11T07:57:33.573374+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_clean_scrub_stamp":"2024-09-11T07:57:07.482151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:53:40.073620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.572573+0000","last_change":"2024-09-11T07:57:10.519531+0000","last_active":"2024-09-11T07:57:33.572573+0000","last_peered":"2024-09-11T07:57:33.572573+0000","last_clean":"2024-09-11T07:57:33.572573+0000","last_became_active":"2024-09-11T07:57:10.519264+0000","last_became_peered":"2024-09-11T07:57:10.519264+0000","last_unstale":"2024-09-11T07:57:33.572573+0000","last_undegraded":"2024-09-11T07:57:33.572573+0000","last_fullsized":"2024-09-11T07:57:33.572573+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_clean_scrub_stamp":"2024-09-11T07:57:09.499431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:12:07.226652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"67'4","reported_seq":24,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.572592+0000","last_change":"2024-09-11T07:57:06.495346+0000","last_active":"2024-09-11T07:57:33.572592+0000","last_peered":"2024-09-11T07:57:33.572592+0000","last_clean":"2024-09-11T07:57:33.572592+0000","last_became_active":"2024-09-11T07:57:06.495085+0000","last_became_peered":"2024-09-11T07:57:06.495085+0000","last_unstale":"2024-09-11T07:57:33.572592+0000","last_undegraded":"2024-09-11T07:57:33.572592+0000","last_fullsized":"2024-09-11T07:57:33.572592+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_clean_scrub_stamp":"2024-09-11T07:57:05.472156+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:00:33.462702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.577072+0000","last_change":"2024-09-11T07:57:04.483251+0000","last_active":"2024-09-11T07:57:33.577072+0000","last_peered":"2024-09-11T07:57:33.577072+0000","last_clean":"2024-09-11T07:57:33.577072+0000","last_became_active":"2024-09-11T07:57:04.482884+0000","last_became_peered":"2024-09-11T07:57:04.482884+0000","last_unstale":"2024-09-11T07:57:33.577072+0000","last_undegraded":"2024-09-11T07:57:33.577072+0000","last_fullsized":"2024-09-11T07:57:33.577072+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:11:39.569390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.611676+0000","last_change":"2024-09-11T07:57:08.512801+0000","last_active":"2024-09-11T07:57:33.611676+0000","last_peered":"2024-09-11T07:57:33.611676+0000","last_clean":"2024-09-11T07:57:33.611676+0000","last_became_active":"2024-09-11T07:57:08.512409+0000","last_became_peered":"2024-09-11T07:57:08.512409+0000","last_unstale":"2024-09-11T07:57:33.611676+0000","last_undegraded":"2024-09-11T07:57:33.611676+0000","last_fullsized":"2024-09-11T07:57:33.611676+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_clean_scrub_stamp":"2024-09-11T07:57:07.482151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:41:49.955075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.573179+0000","last_change":"2024-09-11T07:57:10.521260+0000","last_active":"2024-09-11T07:57:33.573179+0000","last_peered":"2024-09-11T07:57:33.573179+0000","last_clean":"2024-09-11T07:57:33.573179+0000","last_became_active":"2024-09-11T07:57:10.520983+0000","last_became_peered":"2024-09-11T07:57:10.520983+0000","last_unstale":"2024-09-11T07:57:33.573179+0000","last_undegraded":"2024-09-11T07:57:33.573179+0000","last_fullsized":"2024-09-11T07:57:33.573179+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_clean_scrub_stamp":"2024-09-11T07:57:09.499431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:13:23.468974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"67'11","reported_seq":37,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.572812+0000","last_change":"2024-09-11T07:57:06.495975+0000","last_active":"2024-09-11T07:57:33.572812+0000","last_peered":"2024-09-11T07:57:33.572812+0000","last_clean":"2024-09-11T07:57:33.572812+0000","last_became_active":"2024-09-11T07:57:06.495699+0000","last_became_peered":"2024-09-11T07:57:06.495699+0000","last_unstale":"2024-09-11T07:57:33.572812+0000","last_undegraded":"2024-09-11T07:57:33.572812+0000","last_fullsized":"2024-09-11T07:57:33.572812+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_clean_scrub_stamp":"2024-09-11T07:57:05.472156+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:01:29.893432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.612247+0000","last_change":"2024-09-11T07:57:04.481663+0000","last_active":"2024-09-11T07:57:33.612247+0000","last_peered":"2024-09-11T07:57:33.612247+0000","last_clean":"2024-09-11T07:57:33.612247+0000","last_became_active":"2024-09-11T07:57:04.481407+0000","last_became_peered":"2024-09-11T07:57:04.481407+0000","last_unstale":"2024-09-11T07:57:33.612247+0000","last_undegraded":"2024-09-11T07:57:33.612247+0000","last_fullsized":"2024-09-11T07:57:33.612247+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:30:05.991747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.572989+0000","last_change":"2024-09-11T07:57:08.510172+0000","last_active":"2024-09-11T07:57:33.572989+0000","last_peered":"2024-09-11T07:57:33.572989+0000","last_clean":"2024-09-11T07:57:33.572989+0000","last_became_active":"2024-09-11T07:57:08.509878+0000","last_became_peered":"2024-09-11T07:57:08.509878+0000","last_unstale":"2024-09-11T07:57:33.572989+0000","last_undegraded":"2024-09-11T07:57:33.572989+0000","last_fullsized":"2024-09-11T07:57:33.572989+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_clean_scrub_stamp":"2024-09-11T07:57:07.482151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:59:23.968479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.572799+0000","last_change":"2024-09-11T07:57:10.634794+0000","last_active":"2024-09-11T07:57:33.572799+0000","last_peered":"2024-09-11T07:57:33.572799+0000","last_clean":"2024-09-11T07:57:33.572799+0000","last_became_active":"2024-09-11T07:57:10.634405+0000","last_became_peered":"2024-09-11T07:57:10.634405+0000","last_unstale":"2024-09-11T07:57:33.572799+0000","last_undegraded":"2024-09-11T07:57:33.572799+0000","last_fullsized":"2024-09-11T07:57:33.572799+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_clean_scrub_stamp":"2024-09-11T07:57:09.499431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:28:31.501084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"67'1","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.573305+0000","last_change":"2024-09-11T07:57:10.521189+0000","last_active":"2024-09-11T07:57:33.573305+0000","last_peered":"2024-09-11T07:57:33.573305+0000","last_clean":"2024-09-11T07:57:33.573305+0000","last_became_active":"2024-09-11T07:57:10.520922+0000","last_became_peered":"2024-09-11T07:57:10.520922+0000","last_unstale":"2024-09-11T07:57:33.573305+0000","last_undegraded":"2024-09-11T07:57:33.573305+0000","last_fullsized":"2024-09-11T07:57:33.573305+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_clean_scrub_stamp":"2024-09-11T07:57:09.499431+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:06:53.294185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":6,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.576075+0000","last_change":"2024-09-11T07:57:04.488423+0000","last_active":"2024-09-11T07:57:33.576075+0000","last_peered":"2024-09-11T07:57:33.576075+0000","last_clean":"2024-09-11T07:57:33.576075+0000","last_became_active":"2024-09-11T07:57:04.488177+0000","last_became_peered":"2024-09-11T07:57:04.488177+0000","last_unstale":"2024-09-11T07:57:33.576075+0000","last_undegraded":"2024-09-11T07:57:33.576075+0000","last_fullsized":"2024-09-11T07:57:33.576075+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:19:21.070263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"67'10","reported_seq":33,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.611847+0000","last_change":"2024-09-11T07:57:06.494572+0000","last_active":"2024-09-11T07:57:33.611847+0000","last_peered":"2024-09-11T07:57:33.611847+0000","last_clean":"2024-09-11T07:57:33.611847+0000","last_became_active":"2024-09-11T07:57:06.494318+0000","last_became_peered":"2024-09-11T07:57:06.494318+0000","last_unstale":"2024-09-11T07:57:33.611847+0000","last_undegraded":"2024-09-11T07:57:33.611847+0000","last_fullsized":"2024-09-11T07:57:33.611847+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_clean_scrub_stamp":"2024-09-11T07:57:05.472156+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:36:55.588418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"67'11","reported_seq":50,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-11T07:58:37.535328+0000","last_change":"2024-09-11T07:57:08.510208+0000","last_active":"2024-09-11T07:58:37.535328+0000","last_peered":"2024-09-11T07:58:37.535328+0000","last_clean":"2024-09-11T07:58:37.535328+0000","last_became_active":"2024-09-11T07:57:08.508809+0000","last_became_peered":"2024-09-11T07:57:08.508809+0000","last_unstale":"2024-09-11T07:58:37.535328+0000","last_undegraded":"2024-09-11T07:58:37.535328+0000","last_fullsized":"2024-09-11T07:58:37.535328+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_clean_scrub_stamp":"2024-09-11T07:57:07.482151+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:14:46.310751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.573500+0000","last_change":"2024-09-11T07:57:10.513259+0000","last_active":"2024-09-11T07:57:33.573500+0000","last_peered":"2024-09-11T07:57:33.573500+0000","last_clean":"2024-09-11T07:57:33.573500+0000","last_became_active":"2024-09-11T07:57:10.512970+0000","last_became_peered":"2024-09-11T07:57:10.512970+0000","last_unstale":"2024-09-11T07:57:33.573500+0000","last_undegraded":"2024-09-11T07:57:33.573500+0000","last_fullsized":"2024-09-11T07:57:33.573500+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_clean_scrub_stamp":"2024-09-11T07:57:09.499431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:26:30.449501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.573301+0000","last_change":"2024-09-11T07:57:04.486957+0000","last_active":"2024-09-11T07:57:33.573301+0000","last_peered":"2024-09-11T07:57:33.573301+0000","last_clean":"2024-09-11T07:57:33.573301+0000","last_became_active":"2024-09-11T07:57:04.486672+0000","last_became_peered":"2024-09-11T07:57:04.486672+0000","last_unstale":"2024-09-11T07:57:33.573301+0000","last_undegraded":"2024-09-11T07:57:33.573301+0000","last_fullsized":"2024-09-11T07:57:33.573301+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:08:22.841790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"67'11","reported_seq":37,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.612210+0000","last_change":"2024-09-11T07:57:06.500377+0000","last_active":"2024-09-11T07:57:33.612210+0000","last_peered":"2024-09-11T07:57:33.612210+0000","last_clean":"2024-09-11T07:57:33.612210+0000","last_became_active":"2024-09-11T07:57:06.500084+0000","last_became_peered":"2024-09-11T07:57:06.500084+0000","last_unstale":"2024-09-11T07:57:33.612210+0000","last_undegraded":"2024-09-11T07:57:33.612210+0000","last_fullsized":"2024-09-11T07:57:33.612210+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_clean_scrub_stamp":"2024-09-11T07:57:05.472156+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:37:02.139299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.611725+0000","last_change":"2024-09-11T07:57:08.510184+0000","last_active":"2024-09-11T07:57:33.611725+0000","last_peered":"2024-09-11T07:57:33.611725+0000","last_clean":"2024-09-11T07:57:33.611725+0000","last_became_active":"2024-09-11T07:57:08.509696+0000","last_became_peered":"2024-09-11T07:57:08.509696+0000","last_unstale":"2024-09-11T07:57:33.611725+0000","last_undegraded":"2024-09-11T07:57:33.611725+0000","last_fullsized":"2024-09-11T07:57:33.611725+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_clean_scrub_stamp":"2024-09-11T07:57:07.482151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:44:32.552396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.579726+0000","last_change":"2024-09-11T07:57:10.634801+0000","last_active":"2024-09-11T07:57:33.579726+0000","last_peered":"2024-09-11T07:57:33.579726+0000","last_clean":"2024-09-11T07:57:33.579726+0000","last_became_active":"2024-09-11T07:57:10.634465+0000","last_became_peered":"2024-09-11T07:57:10.634465+0000","last_unstale":"2024-09-11T07:57:33.579726+0000","last_undegraded":"2024-09-11T07:57:33.579726+0000","last_fullsized":"2024-09-11T07:57:33.579726+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_clean_scrub_stamp":"2024-09-11T07:57:09.499431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:33:49.467804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.612056+0000","last_change":"2024-09-11T07:57:04.489449+0000","last_active":"2024-09-11T07:57:33.612056+0000","last_peered":"2024-09-11T07:57:33.612056+0000","last_clean":"2024-09-11T07:57:33.612056+0000","last_became_active":"2024-09-11T07:57:04.488395+0000","last_became_peered":"2024-09-11T07:57:04.488395+0000","last_unstale":"2024-09-11T07:57:33.612056+0000","last_undegraded":"2024-09-11T07:57:33.612056+0000","last_fullsized":"2024-09-11T07:57:33.612056+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:34:48.425267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"67'15","reported_seq":69,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.589175+0000","last_change":"2024-09-11T07:57:33.588860+0000","last_active":"2024-09-11T07:57:33.589175+0000","last_peered":"2024-09-11T07:57:33.589175+0000","last_clean":"2024-09-11T07:57:33.589175+0000","last_became_active":"2024-09-11T07:57:33.579410+0000","last_became_peered":"2024-09-11T07:57:33.579410+0000","last_unstale":"2024-09-11T07:57:33.589175+0000","last_undegraded":"2024-09-11T07:57:33.589175+0000","last_fullsized":"2024-09-11T07:57:33.589175+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_clean_scrub_stamp":"2024-09-11T07:57:05.472156+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:01:14.769574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.573759+0000","last_change":"2024-09-11T07:57:08.510016+0000","last_active":"2024-09-11T07:57:33.573759+0000","last_peered":"2024-09-11T07:57:33.573759+0000","last_clean":"2024-09-11T07:57:33.573759+0000","last_became_active":"2024-09-11T07:57:08.508767+0000","last_became_peered":"2024-09-11T07:57:08.508767+0000","last_unstale":"2024-09-11T07:57:33.573759+0000","last_undegraded":"2024-09-11T07:57:33.573759+0000","last_fullsized":"2024-09-11T07:57:33.573759+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_clean_scrub_stamp":"2024-09-11T07:57:07.482151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:01:31.244823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.574564+0000","last_change":"2024-09-11T07:57:10.634192+0000","last_active":"2024-09-11T07:57:33.574564+0000","last_peered":"2024-09-11T07:57:33.574564+0000","last_clean":"2024-09-11T07:57:33.574564+0000","last_became_active":"2024-09-11T07:57:10.633584+0000","last_became_peered":"2024-09-11T07:57:10.633584+0000","last_unstale":"2024-09-11T07:57:33.574564+0000","last_undegraded":"2024-09-11T07:57:33.574564+0000","last_fullsized":"2024-09-11T07:57:33.574564+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_clean_scrub_stamp":"2024-09-11T07:57:09.499431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:30:59.370411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.573399+0000","last_change":"2024-09-11T07:57:04.490493+0000","last_active":"2024-09-11T07:57:33.573399+0000","last_peered":"2024-09-11T07:57:33.573399+0000","last_clean":"2024-09-11T07:57:33.573399+0000","last_became_active":"2024-09-11T07:57:04.490158+0000","last_became_peered":"2024-09-11T07:57:04.490158+0000","last_unstale":"2024-09-11T07:57:33.573399+0000","last_undegraded":"2024-09-11T07:57:33.573399+0000","last_fullsized":"2024-09-11T07:57:33.573399+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:50:43.290557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"67'12","reported_seq":41,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.573991+0000","last_change":"2024-09-11T07:57:06.497329+0000","last_active":"2024-09-11T07:57:33.573991+0000","last_peered":"2024-09-11T07:57:33.573991+0000","last_clean":"2024-09-11T07:57:33.573991+0000","last_became_active":"2024-09-11T07:57:06.497055+0000","last_became_peered":"2024-09-11T07:57:06.497055+0000","last_unstale":"2024-09-11T07:57:33.573991+0000","last_undegraded":"2024-09-11T07:57:33.573991+0000","last_fullsized":"2024-09-11T07:57:33.573991+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_clean_scrub_stamp":"2024-09-11T07:57:05.472156+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:33:49.009590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.579574+0000","last_change":"2024-09-11T07:57:08.499851+0000","last_active":"2024-09-11T07:57:33.579574+0000","last_peered":"2024-09-11T07:57:33.579574+0000","last_clean":"2024-09-11T07:57:33.579574+0000","last_became_active":"2024-09-11T07:57:08.499536+0000","last_became_peered":"2024-09-11T07:57:08.499536+0000","last_unstale":"2024-09-11T07:57:33.579574+0000","last_undegraded":"2024-09-11T07:57:33.579574+0000","last_fullsized":"2024-09-11T07:57:33.579574+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_clean_scrub_stamp":"2024-09-11T07:57:07.482151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:04:22.574399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":6,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88,"ondisk_log_size":88,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":701,"num_read_kb":458,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":578,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":394,"ondisk_log_size":394,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":92,"num_read_kb":87,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":38,"num_read_kb":38,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":156,"num_read_kb":258,"num_write":179,"num_write_kb":3194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234009,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":298708,"kb_used_data":1356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93442348,"statfs":{"total":95990841344,"available":95684964352,"internally_reserved":0,"allocated":1388544,"data_stored":922139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:57:54 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.497}]},{"osd":1,"last update":"Wed Sep 11 07:57:54 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.443},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.46999999999999997}]},{"osd":2,"last update":"Wed Sep 11 07:57:54 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.46000000000000002}]},{"osd":3,"last update":"Wed Sep 11 07:57:54 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.376}]},{"osd":4,"last update":"Wed Sep 11 07:57:54 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.29799999999999999}]},{"osd":5,"last update":"Wed Sep 11 07:57:54 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.313},{"interface":"front","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.42999999999999999}]},{"osd":6,"last update":"Wed Sep 11 07:57:54 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.39600000000000002}]}]},{"osd":6,"up_from":46,"seq":197568495645,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":298692,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93442364,"statfs":{"total":95990841344,"available":95684980736,"internally_reserved":0,"allocated":1372160,"data_stored":921504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:58:31 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.47799999999999998,"15min":0.47599999999999998},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.65100000000000002,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":1.276,"5min":0.66000000000000003,"15min":0.55800000000000005},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":16.039999999999999,"5min":16.039999999999999,"15min":16.039999999999999},"last":0.73899999999999999}]},{"osd":1,"last update":"Wed Sep 11 07:58:31 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.501,"15min":0.505},"min":{"1min":0.376,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.64400000000000002,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.51000000000000001,"15min":0.51700000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.63800000000000001,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.71599999999999997}]},{"osd":2,"last update":"Wed Sep 11 07:58:31 2024","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":0.65100000000000002,"15min":0.54700000000000004},"min":{"1min":0.34699999999999998,"5min":0.309,"15min":0.309},"max":{"1min":16.02,"5min":16.02,"15min":16.02},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.50900000000000001,"15min":0.51500000000000001},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.67200000000000004,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.66200000000000003}]},{"osd":3,"last update":"Wed Sep 11 07:58:31 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.53700000000000003,"15min":0.54700000000000004},"min":{"1min":0.314,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.61899999999999999,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47999999999999998,"15min":0.48199999999999998},"min":{"1min":0.34599999999999997,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.66500000000000004,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.75}]},{"osd":4,"last update":"Wed Sep 11 07:58:31 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.38700000000000001,"15min":0.38600000000000001},"min":{"1min":0.309,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.46800000000000003,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.39900000000000002,"15min":0.39500000000000002},"min":{"1min":0.29699999999999999,"5min":0.246,"15min":0.246},"max":{"1min":0.58499999999999996,"5min":0.627,"15min":0.627},"last":0.54900000000000004}]},{"osd":5,"last update":"Wed Sep 11 07:58:31 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.41099999999999998,"15min":0.40699999999999997},"min":{"1min":0.32200000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.56999999999999995,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.41099999999999998,"15min":0.40400000000000003},"min":{"1min":0.36299999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.61399999999999999,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.58499999999999996}]},{"osd":7,"last update":"Wed Sep 11 07:57:53 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.628}]}]},{"osd":5,"up_from":38,"seq":163208757283,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":298132,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93442924,"statfs":{"total":95990841344,"available":95685554176,"internally_reserved":0,"allocated":794624,"data_stored":331527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:58:04 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.50800000000000001,"15min":0.51000000000000001},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.66800000000000004,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.52100000000000002,"15min":0.52300000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.72399999999999998,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.58199999999999996}]},{"osd":1,"last update":"Wed Sep 11 07:58:04 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.56299999999999994,"15min":0.57699999999999996},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.67600000000000005,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.499,"15min":0.502},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.60799999999999998,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.47599999999999998}]},{"osd":2,"last update":"Wed Sep 11 07:58:04 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.505,"15min":0.50900000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.64600000000000002,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.69699999999999995,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.54000000000000004}]},{"osd":3,"last update":"Wed Sep 11 07:58:04 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.51500000000000001,"15min":0.51800000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.64300000000000002,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.63},{"interface":"front","average":{"1min":0.499,"5min":0.54300000000000004,"15min":0.55100000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.71499999999999997,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.48999999999999999}]},{"osd":4,"last update":"Wed Sep 11 07:58:04 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.378,"15min":0.374},"min":{"1min":0.28199999999999997,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.57599999999999996,"5min":0.59299999999999997,"15min":0.59299999999999997},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.40000000000000002,"15min":0.39800000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.626,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.502}]},{"osd":6,"last update":"Wed Sep 11 07:58:28 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.438,"15min":0.43099999999999999},"min":{"1min":0.31900000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.498,"5min":0.45000000000000001,"15min":0.442},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.64100000000000001,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.67300000000000004}]},{"osd":7,"last update":"Wed Sep 11 07:57:52 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.621}]}]},{"osd":4,"up_from":30,"seq":128849018919,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":298132,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93442924,"statfs":{"total":95990841344,"available":95685554176,"internally_reserved":0,"allocated":794624,"data_stored":331609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:57:41 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.52300000000000002,"15min":0.52700000000000002},"min":{"1min":0.375,"5min":0.309,"15min":0.309},"max":{"1min":0.72099999999999997,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.51900000000000002,"15min":0.52500000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.624,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.42799999999999999}]},{"osd":1,"last update":"Wed Sep 11 07:57:41 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.497,"15min":0.502},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.64300000000000002,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.52500000000000002,"15min":0.52800000000000002},"min":{"1min":0.39600000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.39400000000000002}]},{"osd":2,"last update":"Wed Sep 11 07:57:41 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52000000000000002,"15min":0.51900000000000002},"min":{"1min":0.41699999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.66500000000000004,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.52700000000000002,"15min":0.53600000000000003},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.67400000000000004,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.47399999999999998}]},{"osd":3,"last update":"Wed Sep 11 07:57:41 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.51300000000000001,"15min":0.51500000000000001},"min":{"1min":0.40100000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.66800000000000004,"5min":0.748,"15min":0.748},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.52000000000000002,"15min":0.52600000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.63500000000000001,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.505}]},{"osd":5,"last update":"Wed Sep 11 07:58:04 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.36699999999999999,"15min":0.36299999999999999},"min":{"1min":0.26200000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.59699999999999998,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.433,"5min":0.39600000000000002,"15min":0.39000000000000001},"min":{"1min":0.307,"5min":0.255,"15min":0.255},"max":{"1min":0.629,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.41299999999999998}]},{"osd":6,"last update":"Wed Sep 11 07:58:27 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.38900000000000001,"15min":0.378},"min":{"1min":0.33300000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.437,"15min":0.43099999999999999},"min":{"1min":0.35099999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.54500000000000004}]},{"osd":7,"last update":"Wed Sep 11 07:57:51 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.443},{"interface":"front","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.53700000000000003}]}]},{"osd":3,"up_from":25,"seq":107374182445,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":298148,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93442908,"statfs":{"total":95990841344,"available":95685537792,"internally_reserved":0,"allocated":811008,"data_stored":331611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:58:21 2024","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.36499999999999999,"15min":0.38100000000000001},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.437,"5min":0.56899999999999995,"15min":0.56899999999999995},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.34599999999999997,"5min":0.38600000000000001,"15min":0.40400000000000003},"min":{"1min":0.25600000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.45800000000000002,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.41199999999999998}]},{"osd":1,"last update":"Wed Sep 11 07:58:21 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.40899999999999997,"15min":0.42899999999999999},"min":{"1min":0.29299999999999998,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.52200000000000002,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.38300000000000001,"15min":0.39900000000000002},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.50900000000000001,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.50900000000000001}]},{"osd":2,"last update":"Wed Sep 11 07:58:21 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.42599999999999999,"15min":0.441},"min":{"1min":0.28299999999999997,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.47399999999999998,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.41299999999999998,"15min":0.42899999999999999},"min":{"1min":0.28199999999999997,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.55400000000000005,"5min":0.63,"15min":0.63},"last":0.42499999999999999}]},{"osd":4,"last update":"Wed Sep 11 07:58:39 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":1.0800000000000001,"15min":1.3180000000000001},"min":{"1min":0.39500000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.73899999999999999,"5min":19.266999999999999,"15min":19.266999999999999},"last":0.625},{"interface":"front","average":{"1min":0.501,"5min":1.0700000000000001,"15min":1.3169999999999999},"min":{"1min":0.379,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.64100000000000001,"5min":19.221,"15min":19.221},"last":0.53700000000000003}]},{"osd":5,"last update":"Wed Sep 11 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":1.2190000000000001,"15min":1.337},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.67700000000000005,"5min":19.236000000000001,"15min":19.236000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.504,"5min":0.51300000000000001,"15min":0.51500000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.68600000000000005,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.56399999999999995}]},{"osd":6,"last update":"Wed Sep 11 07:58:28 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53000000000000003,"15min":0.52900000000000003},"min":{"1min":0.36899999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.54300000000000004,"15min":0.54100000000000004},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.69399999999999995,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.59599999999999997}]},{"osd":7,"last update":"Wed Sep 11 07:57:54 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.60599999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411378,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":298112,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93442944,"statfs":{"total":95990841344,"available":95685574656,"internally_reserved":0,"allocated":774144,"data_stored":331560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:57:54 2024","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.34799999999999998,"15min":0.33700000000000002},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.442},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.38500000000000001,"15min":0.37},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.48799999999999999}]},{"osd":1,"last update":"Wed Sep 11 07:57:54 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.33500000000000002,"15min":0.32300000000000001},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.439,"5min":0.35699999999999998,"15min":0.33900000000000002},"min":{"1min":0.28899999999999998,"5min":0.22,"15min":0.22},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.47299999999999998}]},{"osd":3,"last update":"Wed Sep 11 07:58:20 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.42899999999999999,"15min":0.42599999999999999},"min":{"1min":0.29499999999999998,"5min":0.254,"15min":0.254},"max":{"1min":1.651,"5min":1.651,"15min":1.651},"last":0.504},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.40500000000000003,"15min":0.40300000000000002},"min":{"1min":0.27100000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.45600000000000002}]},{"osd":4,"last update":"Wed Sep 11 07:58:39 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.52100000000000002,"15min":0.5},"min":{"1min":0.42399999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.63,"5min":0.55300000000000005,"15min":0.52900000000000003},"min":{"1min":0.40100000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.7669999999999999,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.54900000000000004}]},{"osd":5,"last update":"Wed Sep 11 07:58:04 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.56399999999999995,"15min":0.54600000000000004},"min":{"1min":0.40699999999999997,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.8460000000000001,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.55200000000000005,"15min":0.53600000000000003},"min":{"1min":0.39000000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.716,"5min":1.716,"15min":1.716},"last":0.61199999999999999}]},{"osd":6,"last update":"Wed Sep 11 07:58:30 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.55900000000000005,"15min":0.54500000000000004},"min":{"1min":0.45300000000000001,"5min":0.377,"15min":0.377},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.59199999999999997,"15min":0.57999999999999996},"min":{"1min":0.41999999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.823,"5min":1.823,"15min":1.823},"last":0.628}]},{"osd":7,"last update":"Wed Sep 11 07:57:51 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.7929999999999999,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":0.60199999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574902,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":298140,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93442916,"statfs":{"total":95990841344,"available":95685545984,"internally_reserved":0,"allocated":802816,"data_stored":332704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:58:28 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.34799999999999998,"15min":0.317},"min":{"1min":0.22900000000000001,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.52300000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.249},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.36799999999999999,"15min":0.34100000000000003},"min":{"1min":0.22700000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.25800000000000001}]},{"osd":2,"last update":"Wed Sep 11 07:57:54 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.373,"15min":0.36699999999999999},"min":{"1min":0.26200000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.501,"5min":0.53800000000000003,"15min":0.53800000000000003},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.372,"15min":0.36699999999999999},"min":{"1min":0.247,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.54100000000000004,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.27300000000000002}]},{"osd":3,"last update":"Wed Sep 11 07:58:19 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.41899999999999998,"15min":0.42099999999999999},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.21199999999999999},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.42299999999999999,"15min":0.43099999999999999},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.29899999999999999}]},{"osd":4,"last update":"Wed Sep 11 07:58:40 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53500000000000003,"15min":0.53100000000000003},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.312},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52400000000000002,"15min":0.51400000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.32900000000000001}]},{"osd":5,"last update":"Wed Sep 11 07:58:06 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.54200000000000004,"15min":0.54400000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.55500000000000005,"15min":0.56299999999999994},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.64100000000000001,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.34000000000000002}]},{"osd":6,"last update":"Wed Sep 11 07:58:28 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.55300000000000005,"15min":0.55500000000000005},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.68400000000000005,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56399999999999995,"15min":0.56000000000000005},"min":{"1min":0.45200000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.45700000000000002}]},{"osd":7,"last update":"Wed Sep 11 07:57:54 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.41599999999999998}]}]},{"osd":0,"up_from":8,"seq":34359738427,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":298712,"kb_used_data":1356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93442344,"statfs":{"total":95990841344,"available":95684960256,"internally_reserved":0,"allocated":1388544,"data_stored":922697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 07:58:33 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.33900000000000002,"15min":0.30099999999999999},"min":{"1min":0.25600000000000001,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.34000000000000002,"15min":0.308},"min":{"1min":0.255,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.30299999999999999}]},{"osd":2,"last update":"Wed Sep 11 07:57:54 2024","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.36499999999999999,"15min":0.36099999999999999},"min":{"1min":0.28999999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.496,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.373},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.37,"15min":0.36699999999999999},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.47699999999999998,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.38700000000000001}]},{"osd":3,"last update":"Wed Sep 11 07:58:19 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.41999999999999998,"15min":0.42999999999999999},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.54900000000000004,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.44},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.42099999999999999,"15min":0.42699999999999999},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.40400000000000003}]},{"osd":4,"last update":"Wed Sep 11 07:58:42 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.504,"15min":0.495},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.52800000000000002,"15min":0.52600000000000002},"min":{"1min":0.39000000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.71999999999999997,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.54000000000000004}]},{"osd":5,"last update":"Wed Sep 11 07:58:02 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.39300000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.55200000000000005,"15min":0.55300000000000005},"min":{"1min":0.374,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.59199999999999997}]},{"osd":6,"last update":"Wed Sep 11 07:58:27 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55800000000000005,"15min":0.55900000000000005},"min":{"1min":0.39200000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.72499999999999998,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56999999999999995,"15min":0.56899999999999995},"min":{"1min":0.41799999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.73899999999999999,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.56499999999999995}]},{"osd":7,"last update":"Wed Sep 11 07:57:50 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.502},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.57699999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T07:58:45.493 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T07:58:45.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:45 smithi110 ceph-mon[31709]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:58:45.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:45 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:45.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:45 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:45.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:45 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:45.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:45 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:45.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:45 smithi110 ceph-mon[25392]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:58:45.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:45 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:45.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:45 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:45.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:45 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:45.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:45 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:45.719 INFO:journalctl@ceph.alertmanager.a.smithi110.stdout:Sep 11 07:58:45 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-alertmanager-a[56448]: ts=2024-09-11T07:58:45.665Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-11T07:58:45.719 INFO:journalctl@ceph.alertmanager.a.smithi110.stdout:Sep 11 07:58:45 smithi110 podman[58757]: 2024-09-11 07:58:45.705818976 +0000 UTC m=+0.892799323 container died 818e88a644140a607d86a5219ce6d7037f32e8dcd27e1ab3c695207793bfc733 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T07:58:45.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:45 smithi139 ceph-mon[27311]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:58:45.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:45 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:45.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:45 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:45.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:45 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:45.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:45 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:46.450 INFO:journalctl@ceph.alertmanager.a.smithi110.stdout:Sep 11 07:58:46 smithi110 podman[58757]: 2024-09-11 07:58:46.148325663 +0000 UTC m=+1.335306010 container cleanup 818e88a644140a607d86a5219ce6d7037f32e8dcd27e1ab3c695207793bfc733 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T07:58:46.450 INFO:journalctl@ceph.alertmanager.a.smithi110.stdout:Sep 11 07:58:46 smithi110 bash[58757]: ceph-41bd1096-7012-11ef-bcea-c7b262605968-alertmanager-a 2024-09-11T07:58:46.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:46 smithi110 ceph-mon[31709]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:58:46.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:46 smithi110 ceph-mon[25392]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:58:46.718 INFO:journalctl@ceph.alertmanager.a.smithi110.stdout:Sep 11 07:58:46 smithi110 podman[58897]: 2024-09-11 07:58:46.449584647 +0000 UTC m=+0.746701622 container remove 818e88a644140a607d86a5219ce6d7037f32e8dcd27e1ab3c695207793bfc733 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T07:58:46.718 INFO:journalctl@ceph.alertmanager.a.smithi110.stdout:Sep 11 07:58:46 smithi110 podman[58897]: 2024-09-11 07:58:46.551100044 +0000 UTC m=+0.848217017 volume remove 2d1258838c93fa63c38c973de84387d82af64c13e127ce0d5042628f4aeba493 2024-09-11T07:58:46.748 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:46 smithi139 ceph-mon[27311]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:58:47.227 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:58:47.748 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:47 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:58:47.828 INFO:journalctl@ceph.alertmanager.a.smithi110.stdout:Sep 11 07:58:47 smithi110 podman[59054]: 2024-09-11 07:58:47.641754212 +0000 UTC m=+0.417023756 container died 358243bd5d19e0346ed64ae94acc30bd495465028d716d881a1dbb286ad6ad61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=naughty_colden, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-11T07:58:47.829 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:47 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:58:47.829 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:47 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:58:48.218 INFO:journalctl@ceph.alertmanager.a.smithi110.stdout:Sep 11 07:58:47 smithi110 systemd[1]: ceph-41bd1096-7012-11ef-bcea-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-09-11T07:58:48.218 INFO:journalctl@ceph.alertmanager.a.smithi110.stdout:Sep 11 07:58:47 smithi110 systemd[1]: Stopped Ceph alertmanager.a for 41bd1096-7012-11ef-bcea-c7b262605968. 2024-09-11T07:58:48.218 INFO:journalctl@ceph.alertmanager.a.smithi110.stdout:Sep 11 07:58:47 smithi110 systemd[1]: Starting Ceph alertmanager.a for 41bd1096-7012-11ef-bcea-c7b262605968... 2024-09-11T07:58:48.474 INFO:journalctl@ceph.iscsi.iscsi.a.smithi139.stdout:Sep 11 07:58:48 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-iscsi-iscsi-a[46755]: debug there is no tcmu-runner data available 2024-09-11T07:58:48.678 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:48 smithi110 ceph-mon[31709]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:58:48.679 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:48 smithi110 ceph-mon[25392]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:58:48.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:48 smithi139 ceph-mon[27311]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:58:48.968 INFO:journalctl@ceph.alertmanager.a.smithi110.stdout:Sep 11 07:58:48 smithi110 podman[59193]: 2024-09-11 07:58:48.664756888 +0000 UTC m=+0.175234209 image pull c8568f914cd25b2062c44e9f79f9c18da6e3b85fe0c47a12a2191c61426c2b19 quay.io/prometheus/alertmanager:v0.25.0 2024-09-11T07:58:48.968 INFO:journalctl@ceph.alertmanager.a.smithi110.stdout:Sep 11 07:58:48 smithi110 podman[59193]: 2024-09-11 07:58:48.791319198 +0000 UTC m=+0.301796515 volume create 1a5d84506006c460716148e8a9c36d7a3b80e5c0ab0cb7c759e7196db920c8bc 2024-09-11T07:58:48.968 INFO:journalctl@ceph.alertmanager.a.smithi110.stdout:Sep 11 07:58:48 smithi110 podman[59193]: 2024-09-11 07:58:48.891793779 +0000 UTC m=+0.402271155 container create 304e26c9eb31f6ef695600568713c624c5a05392b2f3cbbbb8426ae43318cfce (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T07:58:49.468 INFO:journalctl@ceph.alertmanager.a.smithi110.stdout:Sep 11 07:58:49 smithi110 podman[59193]: 2024-09-11 07:58:49.195586358 +0000 UTC m=+0.706063672 container init 304e26c9eb31f6ef695600568713c624c5a05392b2f3cbbbb8426ae43318cfce (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T07:58:49.468 INFO:journalctl@ceph.alertmanager.a.smithi110.stdout:Sep 11 07:58:49 smithi110 podman[59193]: 2024-09-11 07:58:49.198451142 +0000 UTC m=+0.708928465 container start 304e26c9eb31f6ef695600568713c624c5a05392b2f3cbbbb8426ae43318cfce (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T07:58:49.469 INFO:journalctl@ceph.alertmanager.a.smithi110.stdout:Sep 11 07:58:49 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-alertmanager-a[59229]: ts=2024-09-11T07:58:49.209Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-09-11T07:58:49.469 INFO:journalctl@ceph.alertmanager.a.smithi110.stdout:Sep 11 07:58:49 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-alertmanager-a[59229]: ts=2024-09-11T07:58:49.210Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-09-11T07:58:49.469 INFO:journalctl@ceph.alertmanager.a.smithi110.stdout:Sep 11 07:58:49 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-alertmanager-a[59229]: ts=2024-09-11T07:58:49.211Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.110 port=9094 2024-09-11T07:58:49.469 INFO:journalctl@ceph.alertmanager.a.smithi110.stdout:Sep 11 07:58:49 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-alertmanager-a[59229]: ts=2024-09-11T07:58:49.211Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-09-11T07:58:49.469 INFO:journalctl@ceph.alertmanager.a.smithi110.stdout:Sep 11 07:58:49 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-alertmanager-a[59229]: ts=2024-09-11T07:58:49.234Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-11T07:58:49.469 INFO:journalctl@ceph.alertmanager.a.smithi110.stdout:Sep 11 07:58:49 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-alertmanager-a[59229]: ts=2024-09-11T07:58:49.235Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-11T07:58:49.470 INFO:journalctl@ceph.alertmanager.a.smithi110.stdout:Sep 11 07:58:49 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-alertmanager-a[59229]: ts=2024-09-11T07:58:49.237Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-09-11T07:58:49.470 INFO:journalctl@ceph.alertmanager.a.smithi110.stdout:Sep 11 07:58:49 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-alertmanager-a[59229]: ts=2024-09-11T07:58:49.237Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-09-11T07:58:49.470 INFO:journalctl@ceph.alertmanager.a.smithi110.stdout:Sep 11 07:58:49 smithi110 bash[59193]: 304e26c9eb31f6ef695600568713c624c5a05392b2f3cbbbb8426ae43318cfce 2024-09-11T07:58:49.470 INFO:journalctl@ceph.alertmanager.a.smithi110.stdout:Sep 11 07:58:49 smithi110 systemd[1]: Started Ceph alertmanager.a for 41bd1096-7012-11ef-bcea-c7b262605968. 2024-09-11T07:58:49.748 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:49 smithi139 ceph-mon[27311]: from='client.24449 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:58:49.748 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:49 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:49.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:49 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:49.752 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T07:58:49.752 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:49 smithi110 ceph-mon[25392]: from='client.24449 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:58:49.752 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:49 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:49.753 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:49 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:49.753 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:49 smithi110 ceph-mon[31709]: from='client.24449 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:58:49.753 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:49 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:49.753 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:49 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:49.758 INFO:teuthology.orchestra.run.smithi110.stderr:dumped all 2024-09-11T07:58:50.595 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:50 smithi110 ceph-mon[25392]: pgmap v34: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:58:50.595 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:50 smithi110 ceph-mon[25392]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-11T07:58:50.595 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:50 smithi110 ceph-mon[25392]: Reconfiguring daemon prometheus.a on smithi139 2024-09-11T07:58:50.595 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:50 smithi110 ceph-mon[31709]: pgmap v34: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:58:50.596 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:50 smithi110 ceph-mon[31709]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-11T07:58:50.596 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:50 smithi110 ceph-mon[31709]: Reconfiguring daemon prometheus.a on smithi139 2024-09-11T07:58:50.597 INFO:teuthology.orchestra.run.smithi110.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2024-09-11T07:58:48.966537+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596188,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":999,"num_read_kb":852,"num_write":609,"num_write_kb":3237,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":1771682,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":635,"ondisk_log_size":635,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749928448,"kb_used":2386776,"kb_used_data":7936,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":747541672,"statfs":{"total":767926730752,"available":765482672128,"internally_reserved":0,"allocated":8126464,"data_stored":4425351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002789"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.572715+0000","last_change":"2024-09-11T07:57:10.635500+0000","last_active":"2024-09-11T07:57:33.572715+0000","last_peered":"2024-09-11T07:57:33.572715+0000","last_clean":"2024-09-11T07:57:33.572715+0000","last_became_active":"2024-09-11T07:57:10.635199+0000","last_became_peered":"2024-09-11T07:57:10.635199+0000","last_unstale":"2024-09-11T07:57:33.572715+0000","last_undegraded":"2024-09-11T07:57:33.572715+0000","last_fullsized":"2024-09-11T07:57:33.572715+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_clean_scrub_stamp":"2024-09-11T07:57:09.499431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:06.145642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"60'1","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.611923+0000","last_change":"2024-09-11T07:57:04.484423+0000","last_active":"2024-09-11T07:57:33.611923+0000","last_peered":"2024-09-11T07:57:33.611923+0000","last_clean":"2024-09-11T07:57:33.611923+0000","last_became_active":"2024-09-11T07:57:04.484049+0000","last_became_peered":"2024-09-11T07:57:04.484049+0000","last_unstale":"2024-09-11T07:57:33.611923+0000","last_undegraded":"2024-09-11T07:57:33.611923+0000","last_fullsized":"2024-09-11T07:57:33.611923+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:15:21.924817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"67'15","reported_seq":43,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.572935+0000","last_change":"2024-09-11T07:57:06.492851+0000","last_active":"2024-09-11T07:57:33.572935+0000","last_peered":"2024-09-11T07:57:33.572935+0000","last_clean":"2024-09-11T07:57:33.572935+0000","last_became_active":"2024-09-11T07:57:06.492565+0000","last_became_peered":"2024-09-11T07:57:06.492565+0000","last_unstale":"2024-09-11T07:57:33.572935+0000","last_undegraded":"2024-09-11T07:57:33.572935+0000","last_fullsized":"2024-09-11T07:57:33.572935+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_clean_scrub_stamp":"2024-09-11T07:57:05.472156+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:43:59.640735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.578960+0000","last_change":"2024-09-11T07:57:08.508110+0000","last_active":"2024-09-11T07:57:33.578960+0000","last_peered":"2024-09-11T07:57:33.578960+0000","last_clean":"2024-09-11T07:57:33.578960+0000","last_became_active":"2024-09-11T07:57:08.507536+0000","last_became_peered":"2024-09-11T07:57:08.507536+0000","last_unstale":"2024-09-11T07:57:33.578960+0000","last_undegraded":"2024-09-11T07:57:33.578960+0000","last_fullsized":"2024-09-11T07:57:33.578960+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_clean_scrub_stamp":"2024-09-11T07:57:07.482151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:17:21.043646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.574406+0000","last_change":"2024-09-11T07:57:04.485662+0000","last_active":"2024-09-11T07:57:33.574406+0000","last_peered":"2024-09-11T07:57:33.574406+0000","last_clean":"2024-09-11T07:57:33.574406+0000","last_became_active":"2024-09-11T07:57:04.485413+0000","last_became_peered":"2024-09-11T07:57:04.485413+0000","last_unstale":"2024-09-11T07:57:33.574406+0000","last_undegraded":"2024-09-11T07:57:33.574406+0000","last_fullsized":"2024-09-11T07:57:33.574406+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:10.763604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"67'9","reported_seq":34,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.578917+0000","last_change":"2024-09-11T07:57:06.498071+0000","last_active":"2024-09-11T07:57:33.578917+0000","last_peered":"2024-09-11T07:57:33.578917+0000","last_clean":"2024-09-11T07:57:33.578917+0000","last_became_active":"2024-09-11T07:57:06.497363+0000","last_became_peered":"2024-09-11T07:57:06.497363+0000","last_unstale":"2024-09-11T07:57:33.578917+0000","last_undegraded":"2024-09-11T07:57:33.578917+0000","last_fullsized":"2024-09-11T07:57:33.578917+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_clean_scrub_stamp":"2024-09-11T07:57:05.472156+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:00:51.725341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.573195+0000","last_change":"2024-09-11T07:57:08.512337+0000","last_active":"2024-09-11T07:57:33.573195+0000","last_peered":"2024-09-11T07:57:33.573195+0000","last_clean":"2024-09-11T07:57:33.573195+0000","last_became_active":"2024-09-11T07:57:08.512058+0000","last_became_peered":"2024-09-11T07:57:08.512058+0000","last_unstale":"2024-09-11T07:57:33.573195+0000","last_undegraded":"2024-09-11T07:57:33.573195+0000","last_fullsized":"2024-09-11T07:57:33.573195+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_clean_scrub_stamp":"2024-09-11T07:57:07.482151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:20:26.339645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.579191+0000","last_change":"2024-09-11T07:57:10.517102+0000","last_active":"2024-09-11T07:57:33.579191+0000","last_peered":"2024-09-11T07:57:33.579191+0000","last_clean":"2024-09-11T07:57:33.579191+0000","last_became_active":"2024-09-11T07:57:10.516696+0000","last_became_peered":"2024-09-11T07:57:10.516696+0000","last_unstale":"2024-09-11T07:57:33.579191+0000","last_undegraded":"2024-09-11T07:57:33.579191+0000","last_fullsized":"2024-09-11T07:57:33.579191+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_clean_scrub_stamp":"2024-09-11T07:57:09.499431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:41:18.372163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.573887+0000","last_change":"2024-09-11T07:57:04.491475+0000","last_active":"2024-09-11T07:57:33.573887+0000","last_peered":"2024-09-11T07:57:33.573887+0000","last_clean":"2024-09-11T07:57:33.573887+0000","last_became_active":"2024-09-11T07:57:04.491229+0000","last_became_peered":"2024-09-11T07:57:04.491229+0000","last_unstale":"2024-09-11T07:57:33.573887+0000","last_undegraded":"2024-09-11T07:57:33.573887+0000","last_fullsized":"2024-09-11T07:57:33.573887+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:29:14.948203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"67'5","reported_seq":28,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.579305+0000","last_change":"2024-09-11T07:57:06.499626+0000","last_active":"2024-09-11T07:57:33.579305+0000","last_peered":"2024-09-11T07:57:33.579305+0000","last_clean":"2024-09-11T07:57:33.579305+0000","last_became_active":"2024-09-11T07:57:06.499336+0000","last_became_peered":"2024-09-11T07:57:06.499336+0000","last_unstale":"2024-09-11T07:57:33.579305+0000","last_undegraded":"2024-09-11T07:57:33.579305+0000","last_fullsized":"2024-09-11T07:57:33.579305+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_clean_scrub_stamp":"2024-09-11T07:57:05.472156+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:37:43.301012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.573106+0000","last_change":"2024-09-11T07:57:08.511340+0000","last_active":"2024-09-11T07:57:33.573106+0000","last_peered":"2024-09-11T07:57:33.573106+0000","last_clean":"2024-09-11T07:57:33.573106+0000","last_became_active":"2024-09-11T07:57:08.511070+0000","last_became_peered":"2024-09-11T07:57:08.511070+0000","last_unstale":"2024-09-11T07:57:33.573106+0000","last_undegraded":"2024-09-11T07:57:33.573106+0000","last_fullsized":"2024-09-11T07:57:33.573106+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_clean_scrub_stamp":"2024-09-11T07:57:07.482151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:28:53.140985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.576117+0000","last_change":"2024-09-11T07:57:10.516638+0000","last_active":"2024-09-11T07:57:33.576117+0000","last_peered":"2024-09-11T07:57:33.576117+0000","last_clean":"2024-09-11T07:57:33.576117+0000","last_became_active":"2024-09-11T07:57:10.516359+0000","last_became_peered":"2024-09-11T07:57:10.516359+0000","last_unstale":"2024-09-11T07:57:33.576117+0000","last_undegraded":"2024-09-11T07:57:33.576117+0000","last_fullsized":"2024-09-11T07:57:33.576117+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_clean_scrub_stamp":"2024-09-11T07:57:09.499431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:21:21.854315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.573887+0000","last_change":"2024-09-11T07:57:04.490240+0000","last_active":"2024-09-11T07:57:33.573887+0000","last_peered":"2024-09-11T07:57:33.573887+0000","last_clean":"2024-09-11T07:57:33.573887+0000","last_became_active":"2024-09-11T07:57:04.489947+0000","last_became_peered":"2024-09-11T07:57:04.489947+0000","last_unstale":"2024-09-11T07:57:33.573887+0000","last_undegraded":"2024-09-11T07:57:33.573887+0000","last_fullsized":"2024-09-11T07:57:33.573887+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:44:17.845923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"67'9","reported_seq":34,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.579540+0000","last_change":"2024-09-11T07:57:06.499163+0000","last_active":"2024-09-11T07:57:33.579540+0000","last_peered":"2024-09-11T07:57:33.579540+0000","last_clean":"2024-09-11T07:57:33.579540+0000","last_became_active":"2024-09-11T07:57:06.498879+0000","last_became_peered":"2024-09-11T07:57:06.498879+0000","last_unstale":"2024-09-11T07:57:33.579540+0000","last_undegraded":"2024-09-11T07:57:33.579540+0000","last_fullsized":"2024-09-11T07:57:33.579540+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_clean_scrub_stamp":"2024-09-11T07:57:05.472156+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:30:26.594241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.576341+0000","last_change":"2024-09-11T07:57:08.507054+0000","last_active":"2024-09-11T07:57:33.576341+0000","last_peered":"2024-09-11T07:57:33.576341+0000","last_clean":"2024-09-11T07:57:33.576341+0000","last_became_active":"2024-09-11T07:57:08.506736+0000","last_became_peered":"2024-09-11T07:57:08.506736+0000","last_unstale":"2024-09-11T07:57:33.576341+0000","last_undegraded":"2024-09-11T07:57:33.576341+0000","last_fullsized":"2024-09-11T07:57:33.576341+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_clean_scrub_stamp":"2024-09-11T07:57:07.482151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:27:04.757009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.611333+0000","last_change":"2024-09-11T07:57:10.519267+0000","last_active":"2024-09-11T07:57:33.611333+0000","last_peered":"2024-09-11T07:57:33.611333+0000","last_clean":"2024-09-11T07:57:33.611333+0000","last_became_active":"2024-09-11T07:57:10.518870+0000","last_became_peered":"2024-09-11T07:57:10.518870+0000","last_unstale":"2024-09-11T07:57:33.611333+0000","last_undegraded":"2024-09-11T07:57:33.611333+0000","last_fullsized":"2024-09-11T07:57:33.611333+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_clean_scrub_stamp":"2024-09-11T07:57:09.499431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:30:22.260019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"67'17","reported_seq":46,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.579873+0000","last_change":"2024-09-11T07:57:06.493823+0000","last_active":"2024-09-11T07:57:33.579873+0000","last_peered":"2024-09-11T07:57:33.579873+0000","last_clean":"2024-09-11T07:57:33.579873+0000","last_became_active":"2024-09-11T07:57:06.493511+0000","last_became_peered":"2024-09-11T07:57:06.493511+0000","last_unstale":"2024-09-11T07:57:33.579873+0000","last_undegraded":"2024-09-11T07:57:33.579873+0000","last_fullsized":"2024-09-11T07:57:33.579873+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_clean_scrub_stamp":"2024-09-11T07:57:05.472156+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:16:59.506222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.573864+0000","last_change":"2024-09-11T07:57:04.489834+0000","last_active":"2024-09-11T07:57:33.573864+0000","last_peered":"2024-09-11T07:57:33.573864+0000","last_clean":"2024-09-11T07:57:33.573864+0000","last_became_active":"2024-09-11T07:57:04.489555+0000","last_became_peered":"2024-09-11T07:57:04.489555+0000","last_unstale":"2024-09-11T07:57:33.573864+0000","last_undegraded":"2024-09-11T07:57:33.573864+0000","last_fullsized":"2024-09-11T07:57:33.573864+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:45:20.906450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.573594+0000","last_change":"2024-09-11T07:57:08.510256+0000","last_active":"2024-09-11T07:57:33.573594+0000","last_peered":"2024-09-11T07:57:33.573594+0000","last_clean":"2024-09-11T07:57:33.573594+0000","last_became_active":"2024-09-11T07:57:08.509077+0000","last_became_peered":"2024-09-11T07:57:08.509077+0000","last_unstale":"2024-09-11T07:57:33.573594+0000","last_undegraded":"2024-09-11T07:57:33.573594+0000","last_fullsized":"2024-09-11T07:57:33.573594+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_clean_scrub_stamp":"2024-09-11T07:57:07.482151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:40:02.315603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.578928+0000","last_change":"2024-09-11T07:57:10.517096+0000","last_active":"2024-09-11T07:57:33.578928+0000","last_peered":"2024-09-11T07:57:33.578928+0000","last_clean":"2024-09-11T07:57:33.578928+0000","last_became_active":"2024-09-11T07:57:10.516818+0000","last_became_peered":"2024-09-11T07:57:10.516818+0000","last_unstale":"2024-09-11T07:57:33.578928+0000","last_undegraded":"2024-09-11T07:57:33.578928+0000","last_fullsized":"2024-09-11T07:57:33.578928+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_clean_scrub_stamp":"2024-09-11T07:57:09.499431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:11:14.042980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"67'10","reported_seq":33,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.579704+0000","last_change":"2024-09-11T07:57:06.498298+0000","last_active":"2024-09-11T07:57:33.579704+0000","last_peered":"2024-09-11T07:57:33.579704+0000","last_clean":"2024-09-11T07:57:33.579704+0000","last_became_active":"2024-09-11T07:57:06.498028+0000","last_became_peered":"2024-09-11T07:57:06.498028+0000","last_unstale":"2024-09-11T07:57:33.579704+0000","last_undegraded":"2024-09-11T07:57:33.579704+0000","last_fullsized":"2024-09-11T07:57:33.579704+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_clean_scrub_stamp":"2024-09-11T07:57:05.472156+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:50:04.600879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.574631+0000","last_change":"2024-09-11T07:57:04.482215+0000","last_active":"2024-09-11T07:57:33.574631+0000","last_peered":"2024-09-11T07:57:33.574631+0000","last_clean":"2024-09-11T07:57:33.574631+0000","last_became_active":"2024-09-11T07:57:04.481911+0000","last_became_peered":"2024-09-11T07:57:04.481911+0000","last_unstale":"2024-09-11T07:57:33.574631+0000","last_undegraded":"2024-09-11T07:57:33.574631+0000","last_fullsized":"2024-09-11T07:57:33.574631+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:52:00.542320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"67'11","reported_seq":48,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-11T07:58:42.536409+0000","last_change":"2024-09-11T07:57:08.504883+0000","last_active":"2024-09-11T07:58:42.536409+0000","last_peered":"2024-09-11T07:58:42.536409+0000","last_clean":"2024-09-11T07:58:42.536409+0000","last_became_active":"2024-09-11T07:57:08.504583+0000","last_became_peered":"2024-09-11T07:57:08.504583+0000","last_unstale":"2024-09-11T07:58:42.536409+0000","last_undegraded":"2024-09-11T07:58:42.536409+0000","last_fullsized":"2024-09-11T07:58:42.536409+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_clean_scrub_stamp":"2024-09-11T07:57:07.482151+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:23:43.798804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.579696+0000","last_change":"2024-09-11T07:57:10.512040+0000","last_active":"2024-09-11T07:57:33.579696+0000","last_peered":"2024-09-11T07:57:33.579696+0000","last_clean":"2024-09-11T07:57:33.579696+0000","last_became_active":"2024-09-11T07:57:10.511769+0000","last_became_peered":"2024-09-11T07:57:10.511769+0000","last_unstale":"2024-09-11T07:57:33.579696+0000","last_undegraded":"2024-09-11T07:57:33.579696+0000","last_fullsized":"2024-09-11T07:57:33.579696+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_clean_scrub_stamp":"2024-09-11T07:57:09.499431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:08:14.606379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"67'15","reported_seq":27,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.606245+0000","last_change":"2024-09-11T07:57:33.606010+0000","last_active":"2024-09-11T07:57:33.606245+0000","last_peered":"2024-09-11T07:57:33.606245+0000","last_clean":"2024-09-11T07:57:33.606245+0000","last_became_active":"2024-09-11T07:57:33.580137+0000","last_became_peered":"2024-09-11T07:57:33.580137+0000","last_unstale":"2024-09-11T07:57:33.606245+0000","last_undegraded":"2024-09-11T07:57:33.606245+0000","last_fullsized":"2024-09-11T07:57:33.606245+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_clean_scrub_stamp":"2024-09-11T07:57:05.472156+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:22:44.605304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.573384+0000","last_change":"2024-09-11T07:57:04.481731+0000","last_active":"2024-09-11T07:57:33.573384+0000","last_peered":"2024-09-11T07:57:33.573384+0000","last_clean":"2024-09-11T07:57:33.573384+0000","last_became_active":"2024-09-11T07:57:04.481457+0000","last_became_peered":"2024-09-11T07:57:04.481457+0000","last_unstale":"2024-09-11T07:57:33.573384+0000","last_undegraded":"2024-09-11T07:57:33.573384+0000","last_fullsized":"2024-09-11T07:57:33.573384+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:53:09.159361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"67'11","reported_seq":48,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-11T07:58:42.536440+0000","last_change":"2024-09-11T07:57:08.503740+0000","last_active":"2024-09-11T07:58:42.536440+0000","last_peered":"2024-09-11T07:58:42.536440+0000","last_clean":"2024-09-11T07:58:42.536440+0000","last_became_active":"2024-09-11T07:57:08.503438+0000","last_became_peered":"2024-09-11T07:57:08.503438+0000","last_unstale":"2024-09-11T07:58:42.536440+0000","last_undegraded":"2024-09-11T07:58:42.536440+0000","last_fullsized":"2024-09-11T07:58:42.536440+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_clean_scrub_stamp":"2024-09-11T07:57:07.482151+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:36:10.473849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.576558+0000","last_change":"2024-09-11T07:57:10.516051+0000","last_active":"2024-09-11T07:57:33.576558+0000","last_peered":"2024-09-11T07:57:33.576558+0000","last_clean":"2024-09-11T07:57:33.576558+0000","last_became_active":"2024-09-11T07:57:10.515581+0000","last_became_peered":"2024-09-11T07:57:10.515581+0000","last_unstale":"2024-09-11T07:57:33.576558+0000","last_undegraded":"2024-09-11T07:57:33.576558+0000","last_fullsized":"2024-09-11T07:57:33.576558+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_clean_scrub_stamp":"2024-09-11T07:57:09.499431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:21:35.149214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"67'11","reported_seq":37,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.579803+0000","last_change":"2024-09-11T07:57:06.499592+0000","last_active":"2024-09-11T07:57:33.579803+0000","last_peered":"2024-09-11T07:57:33.579803+0000","last_clean":"2024-09-11T07:57:33.579803+0000","last_became_active":"2024-09-11T07:57:06.499324+0000","last_became_peered":"2024-09-11T07:57:06.499324+0000","last_unstale":"2024-09-11T07:57:33.579803+0000","last_undegraded":"2024-09-11T07:57:33.579803+0000","last_fullsized":"2024-09-11T07:57:33.579803+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_clean_scrub_stamp":"2024-09-11T07:57:05.472156+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:00:48.941810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.612787+0000","last_change":"2024-09-11T07:57:33.612328+0000","last_active":"2024-09-11T07:57:33.612787+0000","last_peered":"2024-09-11T07:57:33.612787+0000","last_clean":"2024-09-11T07:57:33.612787+0000","last_became_active":"2024-09-11T07:57:33.611979+0000","last_became_peered":"2024-09-11T07:57:33.611979+0000","last_unstale":"2024-09-11T07:57:33.612787+0000","last_undegraded":"2024-09-11T07:57:33.612787+0000","last_fullsized":"2024-09-11T07:57:33.612787+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:31:38.778538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.576936+0000","last_change":"2024-09-11T07:57:08.503430+0000","last_active":"2024-09-11T07:57:33.576936+0000","last_peered":"2024-09-11T07:57:33.576936+0000","last_clean":"2024-09-11T07:57:33.576936+0000","last_became_active":"2024-09-11T07:57:08.503123+0000","last_became_peered":"2024-09-11T07:57:08.503123+0000","last_unstale":"2024-09-11T07:57:33.576936+0000","last_undegraded":"2024-09-11T07:57:33.576936+0000","last_fullsized":"2024-09-11T07:57:33.576936+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_clean_scrub_stamp":"2024-09-11T07:57:07.482151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:02:26.714001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.573057+0000","last_change":"2024-09-11T07:57:10.633968+0000","last_active":"2024-09-11T07:57:33.573057+0000","last_peered":"2024-09-11T07:57:33.573057+0000","last_clean":"2024-09-11T07:57:33.573057+0000","last_became_active":"2024-09-11T07:57:10.633425+0000","last_became_peered":"2024-09-11T07:57:10.633425+0000","last_unstale":"2024-09-11T07:57:33.573057+0000","last_undegraded":"2024-09-11T07:57:33.573057+0000","last_fullsized":"2024-09-11T07:57:33.573057+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_clean_scrub_stamp":"2024-09-11T07:57:09.499431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:32:18.333647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"67'14","reported_seq":39,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.579154+0000","last_change":"2024-09-11T07:57:06.500091+0000","last_active":"2024-09-11T07:57:33.579154+0000","last_peered":"2024-09-11T07:57:33.579154+0000","last_clean":"2024-09-11T07:57:33.579154+0000","last_became_active":"2024-09-11T07:57:06.499802+0000","last_became_peered":"2024-09-11T07:57:06.499802+0000","last_unstale":"2024-09-11T07:57:33.579154+0000","last_undegraded":"2024-09-11T07:57:33.579154+0000","last_fullsized":"2024-09-11T07:57:33.579154+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_clean_scrub_stamp":"2024-09-11T07:57:05.472156+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:46:09.757373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.612262+0000","last_change":"2024-09-11T07:57:04.488374+0000","last_active":"2024-09-11T07:57:33.612262+0000","last_peered":"2024-09-11T07:57:33.612262+0000","last_clean":"2024-09-11T07:57:33.612262+0000","last_became_active":"2024-09-11T07:57:04.488108+0000","last_became_peered":"2024-09-11T07:57:04.488108+0000","last_unstale":"2024-09-11T07:57:33.612262+0000","last_undegraded":"2024-09-11T07:57:33.612262+0000","last_fullsized":"2024-09-11T07:57:33.612262+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:20:32.500248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"67'11","reported_seq":48,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-11T07:58:42.536459+0000","last_change":"2024-09-11T07:57:08.504144+0000","last_active":"2024-09-11T07:58:42.536459+0000","last_peered":"2024-09-11T07:58:42.536459+0000","last_clean":"2024-09-11T07:58:42.536459+0000","last_became_active":"2024-09-11T07:57:08.502566+0000","last_became_peered":"2024-09-11T07:57:08.502566+0000","last_unstale":"2024-09-11T07:58:42.536459+0000","last_undegraded":"2024-09-11T07:58:42.536459+0000","last_fullsized":"2024-09-11T07:58:42.536459+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_clean_scrub_stamp":"2024-09-11T07:57:07.482151+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:26:41.850292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.573477+0000","last_change":"2024-09-11T07:57:10.633883+0000","last_active":"2024-09-11T07:57:33.573477+0000","last_peered":"2024-09-11T07:57:33.573477+0000","last_clean":"2024-09-11T07:57:33.573477+0000","last_became_active":"2024-09-11T07:57:10.633597+0000","last_became_peered":"2024-09-11T07:57:10.633597+0000","last_unstale":"2024-09-11T07:57:33.573477+0000","last_undegraded":"2024-09-11T07:57:33.573477+0000","last_fullsized":"2024-09-11T07:57:33.573477+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_clean_scrub_stamp":"2024-09-11T07:57:09.499431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:27:04.483830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"67'18","reported_seq":50,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.572801+0000","last_change":"2024-09-11T07:57:06.492853+0000","last_active":"2024-09-11T07:57:33.572801+0000","last_peered":"2024-09-11T07:57:33.572801+0000","last_clean":"2024-09-11T07:57:33.572801+0000","last_became_active":"2024-09-11T07:57:06.492556+0000","last_became_peered":"2024-09-11T07:57:06.492556+0000","last_unstale":"2024-09-11T07:57:33.572801+0000","last_undegraded":"2024-09-11T07:57:33.572801+0000","last_fullsized":"2024-09-11T07:57:33.572801+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_clean_scrub_stamp":"2024-09-11T07:57:05.472156+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:08:03.370734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.574370+0000","last_change":"2024-09-11T07:57:04.483733+0000","last_active":"2024-09-11T07:57:33.574370+0000","last_peered":"2024-09-11T07:57:33.574370+0000","last_clean":"2024-09-11T07:57:33.574370+0000","last_became_active":"2024-09-11T07:57:04.483351+0000","last_became_peered":"2024-09-11T07:57:04.483351+0000","last_unstale":"2024-09-11T07:57:33.574370+0000","last_undegraded":"2024-09-11T07:57:33.574370+0000","last_fullsized":"2024-09-11T07:57:33.574370+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:32:09.665437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.579054+0000","last_change":"2024-09-11T07:57:08.505547+0000","last_active":"2024-09-11T07:57:33.579054+0000","last_peered":"2024-09-11T07:57:33.579054+0000","last_clean":"2024-09-11T07:57:33.579054+0000","last_became_active":"2024-09-11T07:57:08.504765+0000","last_became_peered":"2024-09-11T07:57:08.504765+0000","last_unstale":"2024-09-11T07:57:33.579054+0000","last_undegraded":"2024-09-11T07:57:33.579054+0000","last_fullsized":"2024-09-11T07:57:33.579054+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_clean_scrub_stamp":"2024-09-11T07:57:07.482151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:37:32.306648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.579429+0000","last_change":"2024-09-11T07:57:10.516290+0000","last_active":"2024-09-11T07:57:33.579429+0000","last_peered":"2024-09-11T07:57:33.579429+0000","last_clean":"2024-09-11T07:57:33.579429+0000","last_became_active":"2024-09-11T07:57:10.516034+0000","last_became_peered":"2024-09-11T07:57:10.516034+0000","last_unstale":"2024-09-11T07:57:33.579429+0000","last_undegraded":"2024-09-11T07:57:33.579429+0000","last_fullsized":"2024-09-11T07:57:33.579429+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_clean_scrub_stamp":"2024-09-11T07:57:09.499431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:42:38.471181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"67'19","reported_seq":54,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.611699+0000","last_change":"2024-09-11T07:57:06.494748+0000","last_active":"2024-09-11T07:57:33.611699+0000","last_peered":"2024-09-11T07:57:33.611699+0000","last_clean":"2024-09-11T07:57:33.611699+0000","last_became_active":"2024-09-11T07:57:06.493935+0000","last_became_peered":"2024-09-11T07:57:06.493935+0000","last_unstale":"2024-09-11T07:57:33.611699+0000","last_undegraded":"2024-09-11T07:57:33.611699+0000","last_fullsized":"2024-09-11T07:57:33.611699+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_clean_scrub_stamp":"2024-09-11T07:57:05.472156+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:17:29.218282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.573977+0000","last_change":"2024-09-11T07:57:04.491223+0000","last_active":"2024-09-11T07:57:33.573977+0000","last_peered":"2024-09-11T07:57:33.573977+0000","last_clean":"2024-09-11T07:57:33.573977+0000","last_became_active":"2024-09-11T07:57:04.490967+0000","last_became_peered":"2024-09-11T07:57:04.490967+0000","last_unstale":"2024-09-11T07:57:33.573977+0000","last_undegraded":"2024-09-11T07:57:33.573977+0000","last_fullsized":"2024-09-11T07:57:33.573977+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:56:20.746733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.611738+0000","last_change":"2024-09-11T07:57:08.506413+0000","last_active":"2024-09-11T07:57:33.611738+0000","last_peered":"2024-09-11T07:57:33.611738+0000","last_clean":"2024-09-11T07:57:33.611738+0000","last_became_active":"2024-09-11T07:57:08.506108+0000","last_became_peered":"2024-09-11T07:57:08.506108+0000","last_unstale":"2024-09-11T07:57:33.611738+0000","last_undegraded":"2024-09-11T07:57:33.611738+0000","last_fullsized":"2024-09-11T07:57:33.611738+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_clean_scrub_stamp":"2024-09-11T07:57:07.482151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:30:11.388105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.573117+0000","last_change":"2024-09-11T07:57:10.633780+0000","last_active":"2024-09-11T07:57:33.573117+0000","last_peered":"2024-09-11T07:57:33.573117+0000","last_clean":"2024-09-11T07:57:33.573117+0000","last_became_active":"2024-09-11T07:57:10.633413+0000","last_became_peered":"2024-09-11T07:57:10.633413+0000","last_unstale":"2024-09-11T07:57:33.573117+0000","last_undegraded":"2024-09-11T07:57:33.573117+0000","last_fullsized":"2024-09-11T07:57:33.573117+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_clean_scrub_stamp":"2024-09-11T07:57:09.499431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:39:34.058168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"67'10","reported_seq":33,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.574594+0000","last_change":"2024-09-11T07:57:06.496956+0000","last_active":"2024-09-11T07:57:33.574594+0000","last_peered":"2024-09-11T07:57:33.574594+0000","last_clean":"2024-09-11T07:57:33.574594+0000","last_became_active":"2024-09-11T07:57:06.496363+0000","last_became_peered":"2024-09-11T07:57:06.496363+0000","last_unstale":"2024-09-11T07:57:33.574594+0000","last_undegraded":"2024-09-11T07:57:33.574594+0000","last_fullsized":"2024-09-11T07:57:33.574594+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_clean_scrub_stamp":"2024-09-11T07:57:05.472156+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:11:39.417909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.574032+0000","last_change":"2024-09-11T07:57:04.481781+0000","last_active":"2024-09-11T07:57:33.574032+0000","last_peered":"2024-09-11T07:57:33.574032+0000","last_clean":"2024-09-11T07:57:33.574032+0000","last_became_active":"2024-09-11T07:57:04.481523+0000","last_became_peered":"2024-09-11T07:57:04.481523+0000","last_unstale":"2024-09-11T07:57:33.574032+0000","last_undegraded":"2024-09-11T07:57:33.574032+0000","last_fullsized":"2024-09-11T07:57:33.574032+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:01:22.223774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"67'11","reported_seq":48,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-11T07:58:42.536404+0000","last_change":"2024-09-11T07:57:08.506244+0000","last_active":"2024-09-11T07:58:42.536404+0000","last_peered":"2024-09-11T07:58:42.536404+0000","last_clean":"2024-09-11T07:58:42.536404+0000","last_became_active":"2024-09-11T07:57:08.505962+0000","last_became_peered":"2024-09-11T07:57:08.505962+0000","last_unstale":"2024-09-11T07:58:42.536404+0000","last_undegraded":"2024-09-11T07:58:42.536404+0000","last_fullsized":"2024-09-11T07:58:42.536404+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_clean_scrub_stamp":"2024-09-11T07:57:07.482151+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:16:55.639866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.611210+0000","last_change":"2024-09-11T07:57:10.521561+0000","last_active":"2024-09-11T07:57:33.611210+0000","last_peered":"2024-09-11T07:57:33.611210+0000","last_clean":"2024-09-11T07:57:33.611210+0000","last_became_active":"2024-09-11T07:57:10.521023+0000","last_became_peered":"2024-09-11T07:57:10.521023+0000","last_unstale":"2024-09-11T07:57:33.611210+0000","last_undegraded":"2024-09-11T07:57:33.611210+0000","last_fullsized":"2024-09-11T07:57:33.611210+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_clean_scrub_stamp":"2024-09-11T07:57:09.499431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:36:57.173030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"70'31","reported_seq":49,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-11T07:58:42.536442+0000","last_change":"2024-09-11T07:57:33.598370+0000","last_active":"2024-09-11T07:58:42.536442+0000","last_peered":"2024-09-11T07:58:42.536442+0000","last_clean":"2024-09-11T07:58:42.536442+0000","last_became_active":"2024-09-11T07:57:33.580317+0000","last_became_peered":"2024-09-11T07:57:33.580317+0000","last_unstale":"2024-09-11T07:58:42.536442+0000","last_undegraded":"2024-09-11T07:58:42.536442+0000","last_fullsized":"2024-09-11T07:58:42.536442+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_clean_scrub_stamp":"2024-09-11T07:57:05.472156+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:32:56.245828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":51,"num_read_kb":36,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.576462+0000","last_change":"2024-09-11T07:57:04.483428+0000","last_active":"2024-09-11T07:57:33.576462+0000","last_peered":"2024-09-11T07:57:33.576462+0000","last_clean":"2024-09-11T07:57:33.576462+0000","last_became_active":"2024-09-11T07:57:04.482415+0000","last_became_peered":"2024-09-11T07:57:04.482415+0000","last_unstale":"2024-09-11T07:57:33.576462+0000","last_undegraded":"2024-09-11T07:57:33.576462+0000","last_fullsized":"2024-09-11T07:57:33.576462+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:59:07.560339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.611639+0000","last_change":"2024-09-11T07:57:08.508240+0000","last_active":"2024-09-11T07:57:33.611639+0000","last_peered":"2024-09-11T07:57:33.611639+0000","last_clean":"2024-09-11T07:57:33.611639+0000","last_became_active":"2024-09-11T07:57:08.507927+0000","last_became_peered":"2024-09-11T07:57:08.507927+0000","last_unstale":"2024-09-11T07:57:33.611639+0000","last_undegraded":"2024-09-11T07:57:33.611639+0000","last_fullsized":"2024-09-11T07:57:33.611639+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_clean_scrub_stamp":"2024-09-11T07:57:07.482151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:43:56.913115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.573691+0000","last_change":"2024-09-11T07:57:10.517053+0000","last_active":"2024-09-11T07:57:33.573691+0000","last_peered":"2024-09-11T07:57:33.573691+0000","last_clean":"2024-09-11T07:57:33.573691+0000","last_became_active":"2024-09-11T07:57:10.516762+0000","last_became_peered":"2024-09-11T07:57:10.516762+0000","last_unstale":"2024-09-11T07:57:33.573691+0000","last_undegraded":"2024-09-11T07:57:33.573691+0000","last_fullsized":"2024-09-11T07:57:33.573691+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_clean_scrub_stamp":"2024-09-11T07:57:09.499431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:32:50.888196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"67'13","reported_seq":45,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.574101+0000","last_change":"2024-09-11T07:57:06.492607+0000","last_active":"2024-09-11T07:57:33.574101+0000","last_peered":"2024-09-11T07:57:33.574101+0000","last_clean":"2024-09-11T07:57:33.574101+0000","last_became_active":"2024-09-11T07:57:06.492341+0000","last_became_peered":"2024-09-11T07:57:06.492341+0000","last_unstale":"2024-09-11T07:57:33.574101+0000","last_undegraded":"2024-09-11T07:57:33.574101+0000","last_fullsized":"2024-09-11T07:57:33.574101+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_clean_scrub_stamp":"2024-09-11T07:57:05.472156+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:30:00.910172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"67'5","reported_seq":125,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-11T07:58:46.452966+0000","last_change":"2024-09-11T07:57:09.507219+0000","last_active":"2024-09-11T07:58:46.452966+0000","last_peered":"2024-09-11T07:58:46.452966+0000","last_clean":"2024-09-11T07:58:46.452966+0000","last_became_active":"2024-09-11T07:57:04.485435+0000","last_became_peered":"2024-09-11T07:57:04.485435+0000","last_unstale":"2024-09-11T07:58:46.452966+0000","last_undegraded":"2024-09-11T07:58:46.452966+0000","last_fullsized":"2024-09-11T07:58:46.452966+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:23.345662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011489740000000001,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":97,"num_read_kb":92,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.576238+0000","last_change":"2024-09-11T07:57:04.481079+0000","last_active":"2024-09-11T07:57:33.576238+0000","last_peered":"2024-09-11T07:57:33.576238+0000","last_clean":"2024-09-11T07:57:33.576238+0000","last_became_active":"2024-09-11T07:57:04.480774+0000","last_became_peered":"2024-09-11T07:57:04.480774+0000","last_unstale":"2024-09-11T07:57:33.576238+0000","last_undegraded":"2024-09-11T07:57:33.576238+0000","last_fullsized":"2024-09-11T07:57:33.576238+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:55:43.762836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.577379+0000","last_change":"2024-09-11T07:57:08.504850+0000","last_active":"2024-09-11T07:57:33.577379+0000","last_peered":"2024-09-11T07:57:33.577379+0000","last_clean":"2024-09-11T07:57:33.577379+0000","last_became_active":"2024-09-11T07:57:08.504465+0000","last_became_peered":"2024-09-11T07:57:08.504465+0000","last_unstale":"2024-09-11T07:57:33.577379+0000","last_undegraded":"2024-09-11T07:57:33.577379+0000","last_fullsized":"2024-09-11T07:57:33.577379+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_clean_scrub_stamp":"2024-09-11T07:57:07.482151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:06:42.204281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"67'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.573615+0000","last_change":"2024-09-11T07:57:10.635491+0000","last_active":"2024-09-11T07:57:33.573615+0000","last_peered":"2024-09-11T07:57:33.573615+0000","last_clean":"2024-09-11T07:57:33.573615+0000","last_became_active":"2024-09-11T07:57:10.635197+0000","last_became_peered":"2024-09-11T07:57:10.635197+0000","last_unstale":"2024-09-11T07:57:33.573615+0000","last_undegraded":"2024-09-11T07:57:33.573615+0000","last_fullsized":"2024-09-11T07:57:33.573615+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_clean_scrub_stamp":"2024-09-11T07:57:09.499431+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:07:50.407303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"67'12","reported_seq":36,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.611964+0000","last_change":"2024-09-11T07:57:06.495472+0000","last_active":"2024-09-11T07:57:33.611964+0000","last_peered":"2024-09-11T07:57:33.611964+0000","last_clean":"2024-09-11T07:57:33.611964+0000","last_became_active":"2024-09-11T07:57:06.495225+0000","last_became_peered":"2024-09-11T07:57:06.495225+0000","last_unstale":"2024-09-11T07:57:33.611964+0000","last_undegraded":"2024-09-11T07:57:33.611964+0000","last_fullsized":"2024-09-11T07:57:33.611964+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_clean_scrub_stamp":"2024-09-11T07:57:05.472156+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:13:28.038198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"63'1","reported_seq":26,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.611334+0000","last_change":"2024-09-11T07:57:09.506152+0000","last_active":"2024-09-11T07:57:33.611334+0000","last_peered":"2024-09-11T07:57:33.611334+0000","last_clean":"2024-09-11T07:57:33.611334+0000","last_became_active":"2024-09-11T07:57:04.483726+0000","last_became_peered":"2024-09-11T07:57:04.483726+0000","last_unstale":"2024-09-11T07:57:33.611334+0000","last_undegraded":"2024-09-11T07:57:33.611334+0000","last_fullsized":"2024-09-11T07:57:33.611334+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:30:02.189132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011813679999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.573455+0000","last_change":"2024-09-11T07:57:04.492972+0000","last_active":"2024-09-11T07:57:33.573455+0000","last_peered":"2024-09-11T07:57:33.573455+0000","last_clean":"2024-09-11T07:57:33.573455+0000","last_became_active":"2024-09-11T07:57:04.492611+0000","last_became_peered":"2024-09-11T07:57:04.492611+0000","last_unstale":"2024-09-11T07:57:33.573455+0000","last_undegraded":"2024-09-11T07:57:33.573455+0000","last_fullsized":"2024-09-11T07:57:33.573455+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:57:56.550858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.577027+0000","last_change":"2024-09-11T07:57:08.511826+0000","last_active":"2024-09-11T07:57:33.577027+0000","last_peered":"2024-09-11T07:57:33.577027+0000","last_clean":"2024-09-11T07:57:33.577027+0000","last_became_active":"2024-09-11T07:57:08.511505+0000","last_became_peered":"2024-09-11T07:57:08.511505+0000","last_unstale":"2024-09-11T07:57:33.577027+0000","last_undegraded":"2024-09-11T07:57:33.577027+0000","last_fullsized":"2024-09-11T07:57:33.577027+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_clean_scrub_stamp":"2024-09-11T07:57:07.482151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:38:51.881140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.573502+0000","last_change":"2024-09-11T07:57:10.514505+0000","last_active":"2024-09-11T07:57:33.573502+0000","last_peered":"2024-09-11T07:57:33.573502+0000","last_clean":"2024-09-11T07:57:33.573502+0000","last_became_active":"2024-09-11T07:57:10.514044+0000","last_became_peered":"2024-09-11T07:57:10.514044+0000","last_unstale":"2024-09-11T07:57:33.573502+0000","last_undegraded":"2024-09-11T07:57:33.573502+0000","last_fullsized":"2024-09-11T07:57:33.573502+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_clean_scrub_stamp":"2024-09-11T07:57:09.499431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:32:51.813243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"67'14","reported_seq":40,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.612052+0000","last_change":"2024-09-11T07:57:06.497135+0000","last_active":"2024-09-11T07:57:33.612052+0000","last_peered":"2024-09-11T07:57:33.612052+0000","last_clean":"2024-09-11T07:57:33.612052+0000","last_became_active":"2024-09-11T07:57:06.496742+0000","last_became_peered":"2024-09-11T07:57:06.496742+0000","last_unstale":"2024-09-11T07:57:33.612052+0000","last_undegraded":"2024-09-11T07:57:33.612052+0000","last_fullsized":"2024-09-11T07:57:33.612052+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_clean_scrub_stamp":"2024-09-11T07:57:05.472156+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:02.966730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"65'2","reported_seq":27,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.573072+0000","last_change":"2024-09-11T07:57:09.511590+0000","last_active":"2024-09-11T07:57:33.573072+0000","last_peered":"2024-09-11T07:57:33.573072+0000","last_clean":"2024-09-11T07:57:33.573072+0000","last_became_active":"2024-09-11T07:57:04.483268+0000","last_became_peered":"2024-09-11T07:57:04.483268+0000","last_unstale":"2024-09-11T07:57:33.573072+0000","last_undegraded":"2024-09-11T07:57:33.573072+0000","last_fullsized":"2024-09-11T07:57:33.573072+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:39:36.156196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010013400000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.576607+0000","last_change":"2024-09-11T07:57:04.489072+0000","last_active":"2024-09-11T07:57:33.576607+0000","last_peered":"2024-09-11T07:57:33.576607+0000","last_clean":"2024-09-11T07:57:33.576607+0000","last_became_active":"2024-09-11T07:57:04.488800+0000","last_became_peered":"2024-09-11T07:57:04.488800+0000","last_unstale":"2024-09-11T07:57:33.576607+0000","last_undegraded":"2024-09-11T07:57:33.576607+0000","last_fullsized":"2024-09-11T07:57:33.576607+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:03:10.848604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"71'139","reported_seq":273,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-11T07:57:49.027960+0000","last_change":"2024-09-11T07:56:54.440968+0000","last_active":"2024-09-11T07:57:49.027960+0000","last_peered":"2024-09-11T07:57:49.027960+0000","last_clean":"2024-09-11T07:57:49.027960+0000","last_became_active":"2024-09-11T07:56:54.440625+0000","last_became_peered":"2024-09-11T07:56:54.440625+0000","last_unstale":"2024-09-11T07:57:49.027960+0000","last_undegraded":"2024-09-11T07:57:49.027960+0000","last_fullsized":"2024-09-11T07:57:49.027960+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:54:50.621078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:54:50.621078+0000","last_clean_scrub_stamp":"2024-09-11T07:54:50.621078+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:19:00.916869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":156,"num_read_kb":258,"num_write":179,"num_write_kb":3194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.573449+0000","last_change":"2024-09-11T07:57:08.508684+0000","last_active":"2024-09-11T07:57:33.573449+0000","last_peered":"2024-09-11T07:57:33.573449+0000","last_clean":"2024-09-11T07:57:33.573449+0000","last_became_active":"2024-09-11T07:57:08.508410+0000","last_became_peered":"2024-09-11T07:57:08.508410+0000","last_unstale":"2024-09-11T07:57:33.573449+0000","last_undegraded":"2024-09-11T07:57:33.573449+0000","last_fullsized":"2024-09-11T07:57:33.573449+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_clean_scrub_stamp":"2024-09-11T07:57:07.482151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:08.594058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.577051+0000","last_change":"2024-09-11T07:57:10.515110+0000","last_active":"2024-09-11T07:57:33.577051+0000","last_peered":"2024-09-11T07:57:33.577051+0000","last_clean":"2024-09-11T07:57:33.577051+0000","last_became_active":"2024-09-11T07:57:10.514819+0000","last_became_peered":"2024-09-11T07:57:10.514819+0000","last_unstale":"2024-09-11T07:57:33.577051+0000","last_undegraded":"2024-09-11T07:57:33.577051+0000","last_fullsized":"2024-09-11T07:57:33.577051+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_clean_scrub_stamp":"2024-09-11T07:57:09.499431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:59:12.065651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"67'19","reported_seq":49,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.612140+0000","last_change":"2024-09-11T07:57:06.500416+0000","last_active":"2024-09-11T07:57:33.612140+0000","last_peered":"2024-09-11T07:57:33.612140+0000","last_clean":"2024-09-11T07:57:33.612140+0000","last_became_active":"2024-09-11T07:57:06.500023+0000","last_became_peered":"2024-09-11T07:57:06.500023+0000","last_unstale":"2024-09-11T07:57:33.612140+0000","last_undegraded":"2024-09-11T07:57:33.612140+0000","last_fullsized":"2024-09-11T07:57:33.612140+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_clean_scrub_stamp":"2024-09-11T07:57:05.472156+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:36:13.697483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.578870+0000","last_change":"2024-09-11T07:57:04.484400+0000","last_active":"2024-09-11T07:57:33.578870+0000","last_peered":"2024-09-11T07:57:33.578870+0000","last_clean":"2024-09-11T07:57:33.578870+0000","last_became_active":"2024-09-11T07:57:04.484081+0000","last_became_peered":"2024-09-11T07:57:04.484081+0000","last_unstale":"2024-09-11T07:57:33.578870+0000","last_undegraded":"2024-09-11T07:57:33.578870+0000","last_fullsized":"2024-09-11T07:57:33.578870+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:46:59.929296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.577025+0000","last_change":"2024-09-11T07:57:08.506946+0000","last_active":"2024-09-11T07:57:33.577025+0000","last_peered":"2024-09-11T07:57:33.577025+0000","last_clean":"2024-09-11T07:57:33.577025+0000","last_became_active":"2024-09-11T07:57:08.506646+0000","last_became_peered":"2024-09-11T07:57:08.506646+0000","last_unstale":"2024-09-11T07:57:33.577025+0000","last_undegraded":"2024-09-11T07:57:33.577025+0000","last_fullsized":"2024-09-11T07:57:33.577025+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_clean_scrub_stamp":"2024-09-11T07:57:07.482151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:42:42.884334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.573570+0000","last_change":"2024-09-11T07:57:10.521840+0000","last_active":"2024-09-11T07:57:33.573570+0000","last_peered":"2024-09-11T07:57:33.573570+0000","last_clean":"2024-09-11T07:57:33.573570+0000","last_became_active":"2024-09-11T07:57:10.521574+0000","last_became_peered":"2024-09-11T07:57:10.521574+0000","last_unstale":"2024-09-11T07:57:33.573570+0000","last_undegraded":"2024-09-11T07:57:33.573570+0000","last_fullsized":"2024-09-11T07:57:33.573570+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_clean_scrub_stamp":"2024-09-11T07:57:09.499431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:58:43.959712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"67'9","reported_seq":34,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.611586+0000","last_change":"2024-09-11T07:57:06.496305+0000","last_active":"2024-09-11T07:57:33.611586+0000","last_peered":"2024-09-11T07:57:33.611586+0000","last_clean":"2024-09-11T07:57:33.611586+0000","last_became_active":"2024-09-11T07:57:06.495261+0000","last_became_peered":"2024-09-11T07:57:06.495261+0000","last_unstale":"2024-09-11T07:57:33.611586+0000","last_undegraded":"2024-09-11T07:57:33.611586+0000","last_fullsized":"2024-09-11T07:57:33.611586+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_clean_scrub_stamp":"2024-09-11T07:57:05.472156+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:27:24.348733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.574395+0000","last_change":"2024-09-11T07:57:04.484471+0000","last_active":"2024-09-11T07:57:33.574395+0000","last_peered":"2024-09-11T07:57:33.574395+0000","last_clean":"2024-09-11T07:57:33.574395+0000","last_became_active":"2024-09-11T07:57:04.484170+0000","last_became_peered":"2024-09-11T07:57:04.484170+0000","last_unstale":"2024-09-11T07:57:33.574395+0000","last_undegraded":"2024-09-11T07:57:33.574395+0000","last_fullsized":"2024-09-11T07:57:33.574395+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:12:07.946781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.576557+0000","last_change":"2024-09-11T07:57:08.505698+0000","last_active":"2024-09-11T07:57:33.576557+0000","last_peered":"2024-09-11T07:57:33.576557+0000","last_clean":"2024-09-11T07:57:33.576557+0000","last_became_active":"2024-09-11T07:57:08.505391+0000","last_became_peered":"2024-09-11T07:57:08.505391+0000","last_unstale":"2024-09-11T07:57:33.576557+0000","last_undegraded":"2024-09-11T07:57:33.576557+0000","last_fullsized":"2024-09-11T07:57:33.576557+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_clean_scrub_stamp":"2024-09-11T07:57:07.482151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:12:19.857864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.611282+0000","last_change":"2024-09-11T07:57:10.521482+0000","last_active":"2024-09-11T07:57:33.611282+0000","last_peered":"2024-09-11T07:57:33.611282+0000","last_clean":"2024-09-11T07:57:33.611282+0000","last_became_active":"2024-09-11T07:57:10.521053+0000","last_became_peered":"2024-09-11T07:57:10.521053+0000","last_unstale":"2024-09-11T07:57:33.611282+0000","last_undegraded":"2024-09-11T07:57:33.611282+0000","last_fullsized":"2024-09-11T07:57:33.611282+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_clean_scrub_stamp":"2024-09-11T07:57:09.499431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:15:54.897151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"67'15","reported_seq":43,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.575997+0000","last_change":"2024-09-11T07:57:06.494611+0000","last_active":"2024-09-11T07:57:33.575997+0000","last_peered":"2024-09-11T07:57:33.575997+0000","last_clean":"2024-09-11T07:57:33.575997+0000","last_became_active":"2024-09-11T07:57:06.494356+0000","last_became_peered":"2024-09-11T07:57:06.494356+0000","last_unstale":"2024-09-11T07:57:33.575997+0000","last_undegraded":"2024-09-11T07:57:33.575997+0000","last_fullsized":"2024-09-11T07:57:33.575997+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_clean_scrub_stamp":"2024-09-11T07:57:05.472156+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:45:26.230617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.578587+0000","last_change":"2024-09-11T07:57:04.487620+0000","last_active":"2024-09-11T07:57:33.578587+0000","last_peered":"2024-09-11T07:57:33.578587+0000","last_clean":"2024-09-11T07:57:33.578587+0000","last_became_active":"2024-09-11T07:57:04.487356+0000","last_became_peered":"2024-09-11T07:57:04.487356+0000","last_unstale":"2024-09-11T07:57:33.578587+0000","last_undegraded":"2024-09-11T07:57:33.578587+0000","last_fullsized":"2024-09-11T07:57:33.578587+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:44:19.856124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"67'11","reported_seq":48,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-11T07:58:42.536434+0000","last_change":"2024-09-11T07:57:08.512715+0000","last_active":"2024-09-11T07:58:42.536434+0000","last_peered":"2024-09-11T07:58:42.536434+0000","last_clean":"2024-09-11T07:58:42.536434+0000","last_became_active":"2024-09-11T07:57:08.510784+0000","last_became_peered":"2024-09-11T07:57:08.510784+0000","last_unstale":"2024-09-11T07:58:42.536434+0000","last_undegraded":"2024-09-11T07:58:42.536434+0000","last_fullsized":"2024-09-11T07:58:42.536434+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_clean_scrub_stamp":"2024-09-11T07:57:07.482151+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:24:03.571661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.576151+0000","last_change":"2024-09-11T07:57:10.633731+0000","last_active":"2024-09-11T07:57:33.576151+0000","last_peered":"2024-09-11T07:57:33.576151+0000","last_clean":"2024-09-11T07:57:33.576151+0000","last_became_active":"2024-09-11T07:57:10.633431+0000","last_became_peered":"2024-09-11T07:57:10.633431+0000","last_unstale":"2024-09-11T07:57:33.576151+0000","last_undegraded":"2024-09-11T07:57:33.576151+0000","last_fullsized":"2024-09-11T07:57:33.576151+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_clean_scrub_stamp":"2024-09-11T07:57:09.499431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:45:58.755051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"67'12","reported_seq":41,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.579039+0000","last_change":"2024-09-11T07:57:06.498941+0000","last_active":"2024-09-11T07:57:33.579039+0000","last_peered":"2024-09-11T07:57:33.579039+0000","last_clean":"2024-09-11T07:57:33.579039+0000","last_became_active":"2024-09-11T07:57:06.498679+0000","last_became_peered":"2024-09-11T07:57:06.498679+0000","last_unstale":"2024-09-11T07:57:33.579039+0000","last_undegraded":"2024-09-11T07:57:33.579039+0000","last_fullsized":"2024-09-11T07:57:33.579039+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_clean_scrub_stamp":"2024-09-11T07:57:05.472156+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:27:32.243007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"60'2","reported_seq":42,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.580336+0000","last_change":"2024-09-11T07:57:04.484156+0000","last_active":"2024-09-11T07:57:33.580336+0000","last_peered":"2024-09-11T07:57:33.580336+0000","last_clean":"2024-09-11T07:57:33.580336+0000","last_became_active":"2024-09-11T07:57:04.483858+0000","last_became_peered":"2024-09-11T07:57:04.483858+0000","last_unstale":"2024-09-11T07:57:33.580336+0000","last_undegraded":"2024-09-11T07:57:33.580336+0000","last_fullsized":"2024-09-11T07:57:33.580336+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:57:33.680551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.577179+0000","last_change":"2024-09-11T07:57:08.508100+0000","last_active":"2024-09-11T07:57:33.577179+0000","last_peered":"2024-09-11T07:57:33.577179+0000","last_clean":"2024-09-11T07:57:33.577179+0000","last_became_active":"2024-09-11T07:57:08.507823+0000","last_became_peered":"2024-09-11T07:57:08.507823+0000","last_unstale":"2024-09-11T07:57:33.577179+0000","last_undegraded":"2024-09-11T07:57:33.577179+0000","last_fullsized":"2024-09-11T07:57:33.577179+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_clean_scrub_stamp":"2024-09-11T07:57:07.482151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:01:32.478843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.572900+0000","last_change":"2024-09-11T07:57:10.517901+0000","last_active":"2024-09-11T07:57:33.572900+0000","last_peered":"2024-09-11T07:57:33.572900+0000","last_clean":"2024-09-11T07:57:33.572900+0000","last_became_active":"2024-09-11T07:57:10.517630+0000","last_became_peered":"2024-09-11T07:57:10.517630+0000","last_unstale":"2024-09-11T07:57:33.572900+0000","last_undegraded":"2024-09-11T07:57:33.572900+0000","last_fullsized":"2024-09-11T07:57:33.572900+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_clean_scrub_stamp":"2024-09-11T07:57:09.499431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:24:49.152386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"67'9","reported_seq":34,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.611955+0000","last_change":"2024-09-11T07:57:06.494360+0000","last_active":"2024-09-11T07:57:33.611955+0000","last_peered":"2024-09-11T07:57:33.611955+0000","last_clean":"2024-09-11T07:57:33.611955+0000","last_became_active":"2024-09-11T07:57:06.494097+0000","last_became_peered":"2024-09-11T07:57:06.494097+0000","last_unstale":"2024-09-11T07:57:33.611955+0000","last_undegraded":"2024-09-11T07:57:33.611955+0000","last_fullsized":"2024-09-11T07:57:33.611955+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_clean_scrub_stamp":"2024-09-11T07:57:05.472156+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:53:27.619145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"60'1","reported_seq":31,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.577466+0000","last_change":"2024-09-11T07:57:04.481567+0000","last_active":"2024-09-11T07:57:33.577466+0000","last_peered":"2024-09-11T07:57:33.577466+0000","last_clean":"2024-09-11T07:57:33.577466+0000","last_became_active":"2024-09-11T07:57:04.481303+0000","last_became_peered":"2024-09-11T07:57:04.481303+0000","last_unstale":"2024-09-11T07:57:33.577466+0000","last_undegraded":"2024-09-11T07:57:33.577466+0000","last_fullsized":"2024-09-11T07:57:33.577466+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:03:21.719486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.574635+0000","last_change":"2024-09-11T07:57:08.502302+0000","last_active":"2024-09-11T07:57:33.574635+0000","last_peered":"2024-09-11T07:57:33.574635+0000","last_clean":"2024-09-11T07:57:33.574635+0000","last_became_active":"2024-09-11T07:57:08.502033+0000","last_became_peered":"2024-09-11T07:57:08.502033+0000","last_unstale":"2024-09-11T07:57:33.574635+0000","last_undegraded":"2024-09-11T07:57:33.574635+0000","last_fullsized":"2024-09-11T07:57:33.574635+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_clean_scrub_stamp":"2024-09-11T07:57:07.482151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:16:39.551182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.578504+0000","last_change":"2024-09-11T07:57:10.519872+0000","last_active":"2024-09-11T07:57:33.578504+0000","last_peered":"2024-09-11T07:57:33.578504+0000","last_clean":"2024-09-11T07:57:33.578504+0000","last_became_active":"2024-09-11T07:57:10.519607+0000","last_became_peered":"2024-09-11T07:57:10.519607+0000","last_unstale":"2024-09-11T07:57:33.578504+0000","last_undegraded":"2024-09-11T07:57:33.578504+0000","last_fullsized":"2024-09-11T07:57:33.578504+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_clean_scrub_stamp":"2024-09-11T07:57:09.499431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:26:12.890135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.611880+0000","last_change":"2024-09-11T07:57:04.487573+0000","last_active":"2024-09-11T07:57:33.611880+0000","last_peered":"2024-09-11T07:57:33.611880+0000","last_clean":"2024-09-11T07:57:33.611880+0000","last_became_active":"2024-09-11T07:57:04.487303+0000","last_became_peered":"2024-09-11T07:57:04.487303+0000","last_unstale":"2024-09-11T07:57:33.611880+0000","last_undegraded":"2024-09-11T07:57:33.611880+0000","last_fullsized":"2024-09-11T07:57:33.611880+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:14:29.875410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"67'6","reported_seq":27,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.574819+0000","last_change":"2024-09-11T07:57:06.487817+0000","last_active":"2024-09-11T07:57:33.574819+0000","last_peered":"2024-09-11T07:57:33.574819+0000","last_clean":"2024-09-11T07:57:33.574819+0000","last_became_active":"2024-09-11T07:57:06.487555+0000","last_became_peered":"2024-09-11T07:57:06.487555+0000","last_unstale":"2024-09-11T07:57:33.574819+0000","last_undegraded":"2024-09-11T07:57:33.574819+0000","last_fullsized":"2024-09-11T07:57:33.574819+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_clean_scrub_stamp":"2024-09-11T07:57:05.472156+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:17:18.051967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.576824+0000","last_change":"2024-09-11T07:57:08.508247+0000","last_active":"2024-09-11T07:57:33.576824+0000","last_peered":"2024-09-11T07:57:33.576824+0000","last_clean":"2024-09-11T07:57:33.576824+0000","last_became_active":"2024-09-11T07:57:08.507947+0000","last_became_peered":"2024-09-11T07:57:08.507947+0000","last_unstale":"2024-09-11T07:57:33.576824+0000","last_undegraded":"2024-09-11T07:57:33.576824+0000","last_fullsized":"2024-09-11T07:57:33.576824+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_clean_scrub_stamp":"2024-09-11T07:57:07.482151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:14:37.773015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.573385+0000","last_change":"2024-09-11T07:57:10.635467+0000","last_active":"2024-09-11T07:57:33.573385+0000","last_peered":"2024-09-11T07:57:33.573385+0000","last_clean":"2024-09-11T07:57:33.573385+0000","last_became_active":"2024-09-11T07:57:10.635177+0000","last_became_peered":"2024-09-11T07:57:10.635177+0000","last_unstale":"2024-09-11T07:57:33.573385+0000","last_undegraded":"2024-09-11T07:57:33.573385+0000","last_fullsized":"2024-09-11T07:57:33.573385+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_clean_scrub_stamp":"2024-09-11T07:57:09.499431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:09:31.255711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"67'10","reported_seq":33,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.573857+0000","last_change":"2024-09-11T07:57:06.488456+0000","last_active":"2024-09-11T07:57:33.573857+0000","last_peered":"2024-09-11T07:57:33.573857+0000","last_clean":"2024-09-11T07:57:33.573857+0000","last_became_active":"2024-09-11T07:57:06.488206+0000","last_became_peered":"2024-09-11T07:57:06.488206+0000","last_unstale":"2024-09-11T07:57:33.573857+0000","last_undegraded":"2024-09-11T07:57:33.573857+0000","last_fullsized":"2024-09-11T07:57:33.573857+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_clean_scrub_stamp":"2024-09-11T07:57:05.472156+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:36:00.116993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.576537+0000","last_change":"2024-09-11T07:57:04.487787+0000","last_active":"2024-09-11T07:57:33.576537+0000","last_peered":"2024-09-11T07:57:33.576537+0000","last_clean":"2024-09-11T07:57:33.576537+0000","last_became_active":"2024-09-11T07:57:04.487317+0000","last_became_peered":"2024-09-11T07:57:04.487317+0000","last_unstale":"2024-09-11T07:57:33.576537+0000","last_undegraded":"2024-09-11T07:57:33.576537+0000","last_fullsized":"2024-09-11T07:57:33.576537+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:36:55.078580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"67'11","reported_seq":51,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-11T07:58:42.536435+0000","last_change":"2024-09-11T07:57:08.509403+0000","last_active":"2024-09-11T07:58:42.536435+0000","last_peered":"2024-09-11T07:58:42.536435+0000","last_clean":"2024-09-11T07:58:42.536435+0000","last_became_active":"2024-09-11T07:57:08.508994+0000","last_became_peered":"2024-09-11T07:57:08.508994+0000","last_unstale":"2024-09-11T07:58:42.536435+0000","last_undegraded":"2024-09-11T07:58:42.536435+0000","last_fullsized":"2024-09-11T07:58:42.536435+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_clean_scrub_stamp":"2024-09-11T07:57:07.482151+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:43:29.753815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.611546+0000","last_change":"2024-09-11T07:57:10.519763+0000","last_active":"2024-09-11T07:57:33.611546+0000","last_peered":"2024-09-11T07:57:33.611546+0000","last_clean":"2024-09-11T07:57:33.611546+0000","last_became_active":"2024-09-11T07:57:10.519464+0000","last_became_peered":"2024-09-11T07:57:10.519464+0000","last_unstale":"2024-09-11T07:57:33.611546+0000","last_undegraded":"2024-09-11T07:57:33.611546+0000","last_fullsized":"2024-09-11T07:57:33.611546+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_clean_scrub_stamp":"2024-09-11T07:57:09.499431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:10:12.376281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"67'9","reported_seq":34,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.576724+0000","last_change":"2024-09-11T07:57:06.492782+0000","last_active":"2024-09-11T07:57:33.576724+0000","last_peered":"2024-09-11T07:57:33.576724+0000","last_clean":"2024-09-11T07:57:33.576724+0000","last_became_active":"2024-09-11T07:57:06.492521+0000","last_became_peered":"2024-09-11T07:57:06.492521+0000","last_unstale":"2024-09-11T07:57:33.576724+0000","last_undegraded":"2024-09-11T07:57:33.576724+0000","last_fullsized":"2024-09-11T07:57:33.576724+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_clean_scrub_stamp":"2024-09-11T07:57:05.472156+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:59:33.293124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.611847+0000","last_change":"2024-09-11T07:57:04.484744+0000","last_active":"2024-09-11T07:57:33.611847+0000","last_peered":"2024-09-11T07:57:33.611847+0000","last_clean":"2024-09-11T07:57:33.611847+0000","last_became_active":"2024-09-11T07:57:04.483807+0000","last_became_peered":"2024-09-11T07:57:04.483807+0000","last_unstale":"2024-09-11T07:57:33.611847+0000","last_undegraded":"2024-09-11T07:57:33.611847+0000","last_fullsized":"2024-09-11T07:57:33.611847+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:12:27.228249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"67'11","reported_seq":48,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-11T07:58:42.536471+0000","last_change":"2024-09-11T07:57:08.500036+0000","last_active":"2024-09-11T07:58:42.536471+0000","last_peered":"2024-09-11T07:58:42.536471+0000","last_clean":"2024-09-11T07:58:42.536471+0000","last_became_active":"2024-09-11T07:57:08.499732+0000","last_became_peered":"2024-09-11T07:57:08.499732+0000","last_unstale":"2024-09-11T07:58:42.536471+0000","last_undegraded":"2024-09-11T07:58:42.536471+0000","last_fullsized":"2024-09-11T07:58:42.536471+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_clean_scrub_stamp":"2024-09-11T07:57:07.482151+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:04:59.423555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.580100+0000","last_change":"2024-09-11T07:57:10.513630+0000","last_active":"2024-09-11T07:57:33.580100+0000","last_peered":"2024-09-11T07:57:33.580100+0000","last_clean":"2024-09-11T07:57:33.580100+0000","last_became_active":"2024-09-11T07:57:10.513250+0000","last_became_peered":"2024-09-11T07:57:10.513250+0000","last_unstale":"2024-09-11T07:57:33.580100+0000","last_undegraded":"2024-09-11T07:57:33.580100+0000","last_fullsized":"2024-09-11T07:57:33.580100+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_clean_scrub_stamp":"2024-09-11T07:57:09.499431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:06:46.311189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"67'9","reported_seq":34,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.574449+0000","last_change":"2024-09-11T07:57:06.495694+0000","last_active":"2024-09-11T07:57:33.574449+0000","last_peered":"2024-09-11T07:57:33.574449+0000","last_clean":"2024-09-11T07:57:33.574449+0000","last_became_active":"2024-09-11T07:57:06.495434+0000","last_became_peered":"2024-09-11T07:57:06.495434+0000","last_unstale":"2024-09-11T07:57:33.574449+0000","last_undegraded":"2024-09-11T07:57:33.574449+0000","last_fullsized":"2024-09-11T07:57:33.574449+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_clean_scrub_stamp":"2024-09-11T07:57:05.472156+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:14:21.915281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.612110+0000","last_change":"2024-09-11T07:57:04.485350+0000","last_active":"2024-09-11T07:57:33.612110+0000","last_peered":"2024-09-11T07:57:33.612110+0000","last_clean":"2024-09-11T07:57:33.612110+0000","last_became_active":"2024-09-11T07:57:04.485096+0000","last_became_peered":"2024-09-11T07:57:04.485096+0000","last_unstale":"2024-09-11T07:57:33.612110+0000","last_undegraded":"2024-09-11T07:57:33.612110+0000","last_fullsized":"2024-09-11T07:57:33.612110+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:26:19.478481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.572989+0000","last_change":"2024-09-11T07:57:08.501971+0000","last_active":"2024-09-11T07:57:33.572989+0000","last_peered":"2024-09-11T07:57:33.572989+0000","last_clean":"2024-09-11T07:57:33.572989+0000","last_became_active":"2024-09-11T07:57:08.501689+0000","last_became_peered":"2024-09-11T07:57:08.501689+0000","last_unstale":"2024-09-11T07:57:33.572989+0000","last_undegraded":"2024-09-11T07:57:33.572989+0000","last_fullsized":"2024-09-11T07:57:33.572989+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_clean_scrub_stamp":"2024-09-11T07:57:07.482151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:52:21.706527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.611110+0000","last_change":"2024-09-11T07:57:10.515638+0000","last_active":"2024-09-11T07:57:33.611110+0000","last_peered":"2024-09-11T07:57:33.611110+0000","last_clean":"2024-09-11T07:57:33.611110+0000","last_became_active":"2024-09-11T07:57:10.515363+0000","last_became_peered":"2024-09-11T07:57:10.515363+0000","last_unstale":"2024-09-11T07:57:33.611110+0000","last_undegraded":"2024-09-11T07:57:33.611110+0000","last_fullsized":"2024-09-11T07:57:33.611110+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_clean_scrub_stamp":"2024-09-11T07:57:09.499431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:37:06.037292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"67'11","reported_seq":37,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.579479+0000","last_change":"2024-09-11T07:57:06.498266+0000","last_active":"2024-09-11T07:57:33.579479+0000","last_peered":"2024-09-11T07:57:33.579479+0000","last_clean":"2024-09-11T07:57:33.579479+0000","last_became_active":"2024-09-11T07:57:06.497988+0000","last_became_peered":"2024-09-11T07:57:06.497988+0000","last_unstale":"2024-09-11T07:57:33.579479+0000","last_undegraded":"2024-09-11T07:57:33.579479+0000","last_fullsized":"2024-09-11T07:57:33.579479+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_clean_scrub_stamp":"2024-09-11T07:57:05.472156+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:48:58.577105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.574310+0000","last_change":"2024-09-11T07:57:04.480914+0000","last_active":"2024-09-11T07:57:33.574310+0000","last_peered":"2024-09-11T07:57:33.574310+0000","last_clean":"2024-09-11T07:57:33.574310+0000","last_became_active":"2024-09-11T07:57:04.480635+0000","last_became_peered":"2024-09-11T07:57:04.480635+0000","last_unstale":"2024-09-11T07:57:33.574310+0000","last_undegraded":"2024-09-11T07:57:33.574310+0000","last_fullsized":"2024-09-11T07:57:33.574310+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:59:41.799592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.573374+0000","last_change":"2024-09-11T07:57:08.505170+0000","last_active":"2024-09-11T07:57:33.573374+0000","last_peered":"2024-09-11T07:57:33.573374+0000","last_clean":"2024-09-11T07:57:33.573374+0000","last_became_active":"2024-09-11T07:57:08.504855+0000","last_became_peered":"2024-09-11T07:57:08.504855+0000","last_unstale":"2024-09-11T07:57:33.573374+0000","last_undegraded":"2024-09-11T07:57:33.573374+0000","last_fullsized":"2024-09-11T07:57:33.573374+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_clean_scrub_stamp":"2024-09-11T07:57:07.482151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:53:40.073620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.572573+0000","last_change":"2024-09-11T07:57:10.519531+0000","last_active":"2024-09-11T07:57:33.572573+0000","last_peered":"2024-09-11T07:57:33.572573+0000","last_clean":"2024-09-11T07:57:33.572573+0000","last_became_active":"2024-09-11T07:57:10.519264+0000","last_became_peered":"2024-09-11T07:57:10.519264+0000","last_unstale":"2024-09-11T07:57:33.572573+0000","last_undegraded":"2024-09-11T07:57:33.572573+0000","last_fullsized":"2024-09-11T07:57:33.572573+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_clean_scrub_stamp":"2024-09-11T07:57:09.499431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:12:07.226652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"67'4","reported_seq":24,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.572592+0000","last_change":"2024-09-11T07:57:06.495346+0000","last_active":"2024-09-11T07:57:33.572592+0000","last_peered":"2024-09-11T07:57:33.572592+0000","last_clean":"2024-09-11T07:57:33.572592+0000","last_became_active":"2024-09-11T07:57:06.495085+0000","last_became_peered":"2024-09-11T07:57:06.495085+0000","last_unstale":"2024-09-11T07:57:33.572592+0000","last_undegraded":"2024-09-11T07:57:33.572592+0000","last_fullsized":"2024-09-11T07:57:33.572592+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_clean_scrub_stamp":"2024-09-11T07:57:05.472156+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:00:33.462702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.577072+0000","last_change":"2024-09-11T07:57:04.483251+0000","last_active":"2024-09-11T07:57:33.577072+0000","last_peered":"2024-09-11T07:57:33.577072+0000","last_clean":"2024-09-11T07:57:33.577072+0000","last_became_active":"2024-09-11T07:57:04.482884+0000","last_became_peered":"2024-09-11T07:57:04.482884+0000","last_unstale":"2024-09-11T07:57:33.577072+0000","last_undegraded":"2024-09-11T07:57:33.577072+0000","last_fullsized":"2024-09-11T07:57:33.577072+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:11:39.569390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.611676+0000","last_change":"2024-09-11T07:57:08.512801+0000","last_active":"2024-09-11T07:57:33.611676+0000","last_peered":"2024-09-11T07:57:33.611676+0000","last_clean":"2024-09-11T07:57:33.611676+0000","last_became_active":"2024-09-11T07:57:08.512409+0000","last_became_peered":"2024-09-11T07:57:08.512409+0000","last_unstale":"2024-09-11T07:57:33.611676+0000","last_undegraded":"2024-09-11T07:57:33.611676+0000","last_fullsized":"2024-09-11T07:57:33.611676+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_clean_scrub_stamp":"2024-09-11T07:57:07.482151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:41:49.955075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.573179+0000","last_change":"2024-09-11T07:57:10.521260+0000","last_active":"2024-09-11T07:57:33.573179+0000","last_peered":"2024-09-11T07:57:33.573179+0000","last_clean":"2024-09-11T07:57:33.573179+0000","last_became_active":"2024-09-11T07:57:10.520983+0000","last_became_peered":"2024-09-11T07:57:10.520983+0000","last_unstale":"2024-09-11T07:57:33.573179+0000","last_undegraded":"2024-09-11T07:57:33.573179+0000","last_fullsized":"2024-09-11T07:57:33.573179+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_clean_scrub_stamp":"2024-09-11T07:57:09.499431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:13:23.468974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"67'11","reported_seq":37,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.572812+0000","last_change":"2024-09-11T07:57:06.495975+0000","last_active":"2024-09-11T07:57:33.572812+0000","last_peered":"2024-09-11T07:57:33.572812+0000","last_clean":"2024-09-11T07:57:33.572812+0000","last_became_active":"2024-09-11T07:57:06.495699+0000","last_became_peered":"2024-09-11T07:57:06.495699+0000","last_unstale":"2024-09-11T07:57:33.572812+0000","last_undegraded":"2024-09-11T07:57:33.572812+0000","last_fullsized":"2024-09-11T07:57:33.572812+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_clean_scrub_stamp":"2024-09-11T07:57:05.472156+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:01:29.893432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.612247+0000","last_change":"2024-09-11T07:57:04.481663+0000","last_active":"2024-09-11T07:57:33.612247+0000","last_peered":"2024-09-11T07:57:33.612247+0000","last_clean":"2024-09-11T07:57:33.612247+0000","last_became_active":"2024-09-11T07:57:04.481407+0000","last_became_peered":"2024-09-11T07:57:04.481407+0000","last_unstale":"2024-09-11T07:57:33.612247+0000","last_undegraded":"2024-09-11T07:57:33.612247+0000","last_fullsized":"2024-09-11T07:57:33.612247+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:30:05.991747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.572989+0000","last_change":"2024-09-11T07:57:08.510172+0000","last_active":"2024-09-11T07:57:33.572989+0000","last_peered":"2024-09-11T07:57:33.572989+0000","last_clean":"2024-09-11T07:57:33.572989+0000","last_became_active":"2024-09-11T07:57:08.509878+0000","last_became_peered":"2024-09-11T07:57:08.509878+0000","last_unstale":"2024-09-11T07:57:33.572989+0000","last_undegraded":"2024-09-11T07:57:33.572989+0000","last_fullsized":"2024-09-11T07:57:33.572989+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_clean_scrub_stamp":"2024-09-11T07:57:07.482151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:59:23.968479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.572799+0000","last_change":"2024-09-11T07:57:10.634794+0000","last_active":"2024-09-11T07:57:33.572799+0000","last_peered":"2024-09-11T07:57:33.572799+0000","last_clean":"2024-09-11T07:57:33.572799+0000","last_became_active":"2024-09-11T07:57:10.634405+0000","last_became_peered":"2024-09-11T07:57:10.634405+0000","last_unstale":"2024-09-11T07:57:33.572799+0000","last_undegraded":"2024-09-11T07:57:33.572799+0000","last_fullsized":"2024-09-11T07:57:33.572799+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_clean_scrub_stamp":"2024-09-11T07:57:09.499431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:28:31.501084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"67'1","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.573305+0000","last_change":"2024-09-11T07:57:10.521189+0000","last_active":"2024-09-11T07:57:33.573305+0000","last_peered":"2024-09-11T07:57:33.573305+0000","last_clean":"2024-09-11T07:57:33.573305+0000","last_became_active":"2024-09-11T07:57:10.520922+0000","last_became_peered":"2024-09-11T07:57:10.520922+0000","last_unstale":"2024-09-11T07:57:33.573305+0000","last_undegraded":"2024-09-11T07:57:33.573305+0000","last_fullsized":"2024-09-11T07:57:33.573305+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_clean_scrub_stamp":"2024-09-11T07:57:09.499431+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:06:53.294185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":6,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.576075+0000","last_change":"2024-09-11T07:57:04.488423+0000","last_active":"2024-09-11T07:57:33.576075+0000","last_peered":"2024-09-11T07:57:33.576075+0000","last_clean":"2024-09-11T07:57:33.576075+0000","last_became_active":"2024-09-11T07:57:04.488177+0000","last_became_peered":"2024-09-11T07:57:04.488177+0000","last_unstale":"2024-09-11T07:57:33.576075+0000","last_undegraded":"2024-09-11T07:57:33.576075+0000","last_fullsized":"2024-09-11T07:57:33.576075+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:19:21.070263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"67'10","reported_seq":33,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.611847+0000","last_change":"2024-09-11T07:57:06.494572+0000","last_active":"2024-09-11T07:57:33.611847+0000","last_peered":"2024-09-11T07:57:33.611847+0000","last_clean":"2024-09-11T07:57:33.611847+0000","last_became_active":"2024-09-11T07:57:06.494318+0000","last_became_peered":"2024-09-11T07:57:06.494318+0000","last_unstale":"2024-09-11T07:57:33.611847+0000","last_undegraded":"2024-09-11T07:57:33.611847+0000","last_fullsized":"2024-09-11T07:57:33.611847+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_clean_scrub_stamp":"2024-09-11T07:57:05.472156+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:36:55.588418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"67'11","reported_seq":51,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-11T07:58:42.536434+0000","last_change":"2024-09-11T07:57:08.510208+0000","last_active":"2024-09-11T07:58:42.536434+0000","last_peered":"2024-09-11T07:58:42.536434+0000","last_clean":"2024-09-11T07:58:42.536434+0000","last_became_active":"2024-09-11T07:57:08.508809+0000","last_became_peered":"2024-09-11T07:57:08.508809+0000","last_unstale":"2024-09-11T07:58:42.536434+0000","last_undegraded":"2024-09-11T07:58:42.536434+0000","last_fullsized":"2024-09-11T07:58:42.536434+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_clean_scrub_stamp":"2024-09-11T07:57:07.482151+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:14:46.310751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.573500+0000","last_change":"2024-09-11T07:57:10.513259+0000","last_active":"2024-09-11T07:57:33.573500+0000","last_peered":"2024-09-11T07:57:33.573500+0000","last_clean":"2024-09-11T07:57:33.573500+0000","last_became_active":"2024-09-11T07:57:10.512970+0000","last_became_peered":"2024-09-11T07:57:10.512970+0000","last_unstale":"2024-09-11T07:57:33.573500+0000","last_undegraded":"2024-09-11T07:57:33.573500+0000","last_fullsized":"2024-09-11T07:57:33.573500+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_clean_scrub_stamp":"2024-09-11T07:57:09.499431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:26:30.449501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.573301+0000","last_change":"2024-09-11T07:57:04.486957+0000","last_active":"2024-09-11T07:57:33.573301+0000","last_peered":"2024-09-11T07:57:33.573301+0000","last_clean":"2024-09-11T07:57:33.573301+0000","last_became_active":"2024-09-11T07:57:04.486672+0000","last_became_peered":"2024-09-11T07:57:04.486672+0000","last_unstale":"2024-09-11T07:57:33.573301+0000","last_undegraded":"2024-09-11T07:57:33.573301+0000","last_fullsized":"2024-09-11T07:57:33.573301+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:08:22.841790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"67'11","reported_seq":37,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.612210+0000","last_change":"2024-09-11T07:57:06.500377+0000","last_active":"2024-09-11T07:57:33.612210+0000","last_peered":"2024-09-11T07:57:33.612210+0000","last_clean":"2024-09-11T07:57:33.612210+0000","last_became_active":"2024-09-11T07:57:06.500084+0000","last_became_peered":"2024-09-11T07:57:06.500084+0000","last_unstale":"2024-09-11T07:57:33.612210+0000","last_undegraded":"2024-09-11T07:57:33.612210+0000","last_fullsized":"2024-09-11T07:57:33.612210+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_clean_scrub_stamp":"2024-09-11T07:57:05.472156+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:37:02.139299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.611725+0000","last_change":"2024-09-11T07:57:08.510184+0000","last_active":"2024-09-11T07:57:33.611725+0000","last_peered":"2024-09-11T07:57:33.611725+0000","last_clean":"2024-09-11T07:57:33.611725+0000","last_became_active":"2024-09-11T07:57:08.509696+0000","last_became_peered":"2024-09-11T07:57:08.509696+0000","last_unstale":"2024-09-11T07:57:33.611725+0000","last_undegraded":"2024-09-11T07:57:33.611725+0000","last_fullsized":"2024-09-11T07:57:33.611725+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_clean_scrub_stamp":"2024-09-11T07:57:07.482151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:44:32.552396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.579726+0000","last_change":"2024-09-11T07:57:10.634801+0000","last_active":"2024-09-11T07:57:33.579726+0000","last_peered":"2024-09-11T07:57:33.579726+0000","last_clean":"2024-09-11T07:57:33.579726+0000","last_became_active":"2024-09-11T07:57:10.634465+0000","last_became_peered":"2024-09-11T07:57:10.634465+0000","last_unstale":"2024-09-11T07:57:33.579726+0000","last_undegraded":"2024-09-11T07:57:33.579726+0000","last_fullsized":"2024-09-11T07:57:33.579726+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_clean_scrub_stamp":"2024-09-11T07:57:09.499431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:33:49.467804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.612056+0000","last_change":"2024-09-11T07:57:04.489449+0000","last_active":"2024-09-11T07:57:33.612056+0000","last_peered":"2024-09-11T07:57:33.612056+0000","last_clean":"2024-09-11T07:57:33.612056+0000","last_became_active":"2024-09-11T07:57:04.488395+0000","last_became_peered":"2024-09-11T07:57:04.488395+0000","last_unstale":"2024-09-11T07:57:33.612056+0000","last_undegraded":"2024-09-11T07:57:33.612056+0000","last_fullsized":"2024-09-11T07:57:33.612056+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:34:48.425267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"67'15","reported_seq":69,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.589175+0000","last_change":"2024-09-11T07:57:33.588860+0000","last_active":"2024-09-11T07:57:33.589175+0000","last_peered":"2024-09-11T07:57:33.589175+0000","last_clean":"2024-09-11T07:57:33.589175+0000","last_became_active":"2024-09-11T07:57:33.579410+0000","last_became_peered":"2024-09-11T07:57:33.579410+0000","last_unstale":"2024-09-11T07:57:33.589175+0000","last_undegraded":"2024-09-11T07:57:33.589175+0000","last_fullsized":"2024-09-11T07:57:33.589175+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_clean_scrub_stamp":"2024-09-11T07:57:05.472156+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:01:14.769574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.573759+0000","last_change":"2024-09-11T07:57:08.510016+0000","last_active":"2024-09-11T07:57:33.573759+0000","last_peered":"2024-09-11T07:57:33.573759+0000","last_clean":"2024-09-11T07:57:33.573759+0000","last_became_active":"2024-09-11T07:57:08.508767+0000","last_became_peered":"2024-09-11T07:57:08.508767+0000","last_unstale":"2024-09-11T07:57:33.573759+0000","last_undegraded":"2024-09-11T07:57:33.573759+0000","last_fullsized":"2024-09-11T07:57:33.573759+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_clean_scrub_stamp":"2024-09-11T07:57:07.482151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:01:31.244823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.574564+0000","last_change":"2024-09-11T07:57:10.634192+0000","last_active":"2024-09-11T07:57:33.574564+0000","last_peered":"2024-09-11T07:57:33.574564+0000","last_clean":"2024-09-11T07:57:33.574564+0000","last_became_active":"2024-09-11T07:57:10.633584+0000","last_became_peered":"2024-09-11T07:57:10.633584+0000","last_unstale":"2024-09-11T07:57:33.574564+0000","last_undegraded":"2024-09-11T07:57:33.574564+0000","last_fullsized":"2024-09-11T07:57:33.574564+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:09.499431+0000","last_clean_scrub_stamp":"2024-09-11T07:57:09.499431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:30:59.370411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.573399+0000","last_change":"2024-09-11T07:57:04.490493+0000","last_active":"2024-09-11T07:57:33.573399+0000","last_peered":"2024-09-11T07:57:33.573399+0000","last_clean":"2024-09-11T07:57:33.573399+0000","last_became_active":"2024-09-11T07:57:04.490158+0000","last_became_peered":"2024-09-11T07:57:04.490158+0000","last_unstale":"2024-09-11T07:57:33.573399+0000","last_undegraded":"2024-09-11T07:57:33.573399+0000","last_fullsized":"2024-09-11T07:57:33.573399+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:03.462382+0000","last_clean_scrub_stamp":"2024-09-11T07:57:03.462382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:50:43.290557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"67'12","reported_seq":41,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.573991+0000","last_change":"2024-09-11T07:57:06.497329+0000","last_active":"2024-09-11T07:57:33.573991+0000","last_peered":"2024-09-11T07:57:33.573991+0000","last_clean":"2024-09-11T07:57:33.573991+0000","last_became_active":"2024-09-11T07:57:06.497055+0000","last_became_peered":"2024-09-11T07:57:06.497055+0000","last_unstale":"2024-09-11T07:57:33.573991+0000","last_undegraded":"2024-09-11T07:57:33.573991+0000","last_fullsized":"2024-09-11T07:57:33.573991+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:05.472156+0000","last_clean_scrub_stamp":"2024-09-11T07:57:05.472156+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:33:49.009590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-11T07:57:33.579574+0000","last_change":"2024-09-11T07:57:08.499851+0000","last_active":"2024-09-11T07:57:33.579574+0000","last_peered":"2024-09-11T07:57:33.579574+0000","last_clean":"2024-09-11T07:57:33.579574+0000","last_became_active":"2024-09-11T07:57:08.499536+0000","last_became_peered":"2024-09-11T07:57:08.499536+0000","last_unstale":"2024-09-11T07:57:33.579574+0000","last_undegraded":"2024-09-11T07:57:33.579574+0000","last_fullsized":"2024-09-11T07:57:33.579574+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:57:07.482151+0000","last_clean_scrub_stamp":"2024-09-11T07:57:07.482151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:04:22.574399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":6,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88,"ondisk_log_size":88,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":701,"num_read_kb":458,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":578,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":394,"ondisk_log_size":394,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":97,"num_read_kb":92,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":38,"num_read_kb":38,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":156,"num_read_kb":258,"num_write":179,"num_write_kb":3194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234010,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":298708,"kb_used_data":1356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93442348,"statfs":{"total":95990841344,"available":95684964352,"internally_reserved":0,"allocated":1388544,"data_stored":922139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:57:54 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.62},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.66000000000000003}]},{"osd":1,"last update":"Wed Sep 11 07:57:54 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.58799999999999997}]},{"osd":2,"last update":"Wed Sep 11 07:57:54 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.45100000000000001}]},{"osd":3,"last update":"Wed Sep 11 07:57:54 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.64500000000000002}]},{"osd":4,"last update":"Wed Sep 11 07:57:54 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.56200000000000006}]},{"osd":5,"last update":"Wed Sep 11 07:57:54 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.72899999999999998}]},{"osd":6,"last update":"Wed Sep 11 07:57:54 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.63400000000000001}]}]},{"osd":6,"up_from":46,"seq":197568495646,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":298692,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93442364,"statfs":{"total":95990841344,"available":95684980736,"internally_reserved":0,"allocated":1372160,"data_stored":921504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:58:31 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.47799999999999998,"15min":0.47599999999999998},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.65100000000000002,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.438},{"interface":"front","average":{"1min":1.276,"5min":0.66000000000000003,"15min":0.55800000000000005},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":16.039999999999999,"5min":16.039999999999999,"15min":16.039999999999999},"last":0.52400000000000002}]},{"osd":1,"last update":"Wed Sep 11 07:58:31 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.501,"15min":0.505},"min":{"1min":0.376,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.64400000000000002,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.51000000000000001,"15min":0.51700000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.63800000000000001,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.48999999999999999}]},{"osd":2,"last update":"Wed Sep 11 07:58:31 2024","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":0.65100000000000002,"15min":0.54700000000000004},"min":{"1min":0.34699999999999998,"5min":0.309,"15min":0.309},"max":{"1min":16.02,"5min":16.02,"15min":16.02},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.50900000000000001,"15min":0.51500000000000001},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.67200000000000004,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.34399999999999997}]},{"osd":3,"last update":"Wed Sep 11 07:58:31 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.53700000000000003,"15min":0.54700000000000004},"min":{"1min":0.314,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.61899999999999999,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47999999999999998,"15min":0.48199999999999998},"min":{"1min":0.34599999999999997,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.66500000000000004,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.39100000000000001}]},{"osd":4,"last update":"Wed Sep 11 07:58:31 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.38700000000000001,"15min":0.38600000000000001},"min":{"1min":0.309,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.46800000000000003,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.39900000000000002,"15min":0.39500000000000002},"min":{"1min":0.29699999999999999,"5min":0.246,"15min":0.246},"max":{"1min":0.58499999999999996,"5min":0.627,"15min":0.627},"last":0.317}]},{"osd":5,"last update":"Wed Sep 11 07:58:31 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.41099999999999998,"15min":0.40699999999999997},"min":{"1min":0.32200000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.56999999999999995,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.41099999999999998,"15min":0.40400000000000003},"min":{"1min":0.36299999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.61399999999999999,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.504}]},{"osd":7,"last update":"Wed Sep 11 07:57:53 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.55300000000000005}]}]},{"osd":5,"up_from":38,"seq":163208757284,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":298132,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93442924,"statfs":{"total":95990841344,"available":95685554176,"internally_reserved":0,"allocated":794624,"data_stored":331527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:58:04 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.50800000000000001,"15min":0.51000000000000001},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.66800000000000004,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.52100000000000002,"15min":0.52300000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.72399999999999998,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.42899999999999999}]},{"osd":1,"last update":"Wed Sep 11 07:58:04 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.56299999999999994,"15min":0.57699999999999996},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.67600000000000005,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.499,"15min":0.502},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.60799999999999998,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.41099999999999998}]},{"osd":2,"last update":"Wed Sep 11 07:58:04 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.505,"15min":0.50900000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.64600000000000002,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.69699999999999995,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.502}]},{"osd":3,"last update":"Wed Sep 11 07:58:04 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.51500000000000001,"15min":0.51800000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.64300000000000002,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.54300000000000004,"15min":0.55100000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.71499999999999997,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.45400000000000001}]},{"osd":4,"last update":"Wed Sep 11 07:58:04 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.378,"15min":0.374},"min":{"1min":0.28199999999999997,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.57599999999999996,"5min":0.59299999999999997,"15min":0.59299999999999997},"last":0.379},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.40000000000000002,"15min":0.39800000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.626,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.308}]},{"osd":6,"last update":"Wed Sep 11 07:58:28 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.438,"15min":0.43099999999999999},"min":{"1min":0.31900000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.372},{"interface":"front","average":{"1min":0.498,"5min":0.45000000000000001,"15min":0.442},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.64100000000000001,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.47899999999999998}]},{"osd":7,"last update":"Wed Sep 11 07:57:52 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.32100000000000001}]}]},{"osd":4,"up_from":30,"seq":128849018920,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":298132,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93442924,"statfs":{"total":95990841344,"available":95685554176,"internally_reserved":0,"allocated":794624,"data_stored":331609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:58:44 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.51800000000000002,"15min":0.52600000000000002},"min":{"1min":0.35599999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.66000000000000003,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51700000000000002,"15min":0.52400000000000002},"min":{"1min":0.33800000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.69299999999999995,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.56499999999999995}]},{"osd":1,"last update":"Wed Sep 11 07:58:44 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.496,"15min":0.502},"min":{"1min":0.31900000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.66300000000000003,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.52600000000000002,"15min":0.52800000000000002},"min":{"1min":0.39400000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.67700000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.59399999999999997}]},{"osd":2,"last update":"Wed Sep 11 07:58:44 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.52600000000000002,"15min":0.52100000000000002},"min":{"1min":0.45400000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.64100000000000001,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.52300000000000002,"15min":0.53500000000000003},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.65100000000000002,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.51900000000000002}]},{"osd":3,"last update":"Wed Sep 11 07:58:44 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.51400000000000001,"15min":0.51500000000000001},"min":{"1min":0.40600000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.67800000000000005,"5min":0.748,"15min":0.748},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.51900000000000002,"15min":0.52500000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.58399999999999996}]},{"osd":5,"last update":"Wed Sep 11 07:58:04 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.36699999999999999,"15min":0.36299999999999999},"min":{"1min":0.26200000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.59699999999999998,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.433,"5min":0.39600000000000002,"15min":0.39000000000000001},"min":{"1min":0.307,"5min":0.255,"15min":0.255},"max":{"1min":0.629,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.495}]},{"osd":6,"last update":"Wed Sep 11 07:58:27 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.38900000000000001,"15min":0.378},"min":{"1min":0.33300000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.44},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.437,"15min":0.43099999999999999},"min":{"1min":0.35099999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.55600000000000005}]},{"osd":7,"last update":"Wed Sep 11 07:57:51 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.54400000000000004}]}]},{"osd":3,"up_from":25,"seq":107374182446,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":298148,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93442908,"statfs":{"total":95990841344,"available":95685537792,"internally_reserved":0,"allocated":811008,"data_stored":331611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:58:21 2024","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.36499999999999999,"15min":0.38100000000000001},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.437,"5min":0.56899999999999995,"15min":0.56899999999999995},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.34599999999999997,"5min":0.38600000000000001,"15min":0.40400000000000003},"min":{"1min":0.25600000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.45800000000000002,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.27000000000000002}]},{"osd":1,"last update":"Wed Sep 11 07:58:21 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.40899999999999997,"15min":0.42899999999999999},"min":{"1min":0.29299999999999998,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.52200000000000002,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.38300000000000001,"15min":0.39900000000000002},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.50900000000000001,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.28199999999999997}]},{"osd":2,"last update":"Wed Sep 11 07:58:21 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.42599999999999999,"15min":0.441},"min":{"1min":0.28299999999999997,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.47399999999999998,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.41299999999999998,"15min":0.42899999999999999},"min":{"1min":0.28199999999999997,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.55400000000000005,"5min":0.63,"15min":0.63},"last":0.253}]},{"osd":4,"last update":"Wed Sep 11 07:58:39 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":1.0800000000000001,"15min":1.3180000000000001},"min":{"1min":0.39500000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.73899999999999999,"5min":19.266999999999999,"15min":19.266999999999999},"last":0.502},{"interface":"front","average":{"1min":0.501,"5min":1.0700000000000001,"15min":1.3169999999999999},"min":{"1min":0.379,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.64100000000000001,"5min":19.221,"15min":19.221},"last":0.46100000000000002}]},{"osd":5,"last update":"Wed Sep 11 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":1.2190000000000001,"15min":1.337},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.67700000000000005,"5min":19.236000000000001,"15min":19.236000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.504,"5min":0.51300000000000001,"15min":0.51500000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.68600000000000005,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.438}]},{"osd":6,"last update":"Wed Sep 11 07:58:28 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53000000000000003,"15min":0.52900000000000003},"min":{"1min":0.36899999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.54300000000000004,"15min":0.54100000000000004},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.69399999999999995,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.48999999999999999}]},{"osd":7,"last update":"Wed Sep 11 07:57:54 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.47699999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411379,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":298112,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93442944,"statfs":{"total":95990841344,"available":95685574656,"internally_reserved":0,"allocated":774144,"data_stored":331560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:57:54 2024","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.34799999999999998,"15min":0.33700000000000002},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.219},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.38500000000000001,"15min":0.37},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.309}]},{"osd":1,"last update":"Wed Sep 11 07:57:54 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.33500000000000002,"15min":0.32300000000000001},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.27000000000000002},{"interface":"front","average":{"1min":0.439,"5min":0.35699999999999998,"15min":0.33900000000000002},"min":{"1min":0.28899999999999998,"5min":0.22,"15min":0.22},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.33500000000000002}]},{"osd":3,"last update":"Wed Sep 11 07:58:20 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.42899999999999999,"15min":0.42599999999999999},"min":{"1min":0.29499999999999998,"5min":0.254,"15min":0.254},"max":{"1min":1.651,"5min":1.651,"15min":1.651},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.40500000000000003,"15min":0.40300000000000002},"min":{"1min":0.27100000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.253}]},{"osd":4,"last update":"Wed Sep 11 07:58:39 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.52100000000000002,"15min":0.5},"min":{"1min":0.42399999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.63,"5min":0.55300000000000005,"15min":0.52900000000000003},"min":{"1min":0.40100000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.7669999999999999,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.69999999999999996}]},{"osd":5,"last update":"Wed Sep 11 07:58:04 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.56399999999999995,"15min":0.54600000000000004},"min":{"1min":0.40699999999999997,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.8460000000000001,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":0.751},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.55200000000000005,"15min":0.53600000000000003},"min":{"1min":0.39000000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.716,"5min":1.716,"15min":1.716},"last":0.56699999999999995}]},{"osd":6,"last update":"Wed Sep 11 07:58:30 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.55900000000000005,"15min":0.54500000000000004},"min":{"1min":0.45300000000000001,"5min":0.377,"15min":0.377},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.59199999999999997,"15min":0.57999999999999996},"min":{"1min":0.41999999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.823,"5min":1.823,"15min":1.823},"last":0.68100000000000005}]},{"osd":7,"last update":"Wed Sep 11 07:57:51 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.7929999999999999,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":0.57799999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574903,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":298140,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93442916,"statfs":{"total":95990841344,"available":95685545984,"internally_reserved":0,"allocated":802816,"data_stored":332704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:58:28 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.34799999999999998,"15min":0.317},"min":{"1min":0.22900000000000001,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.52300000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.36799999999999999,"15min":0.34100000000000003},"min":{"1min":0.22700000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.34000000000000002}]},{"osd":2,"last update":"Wed Sep 11 07:57:54 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.373,"15min":0.36699999999999999},"min":{"1min":0.26200000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.501,"5min":0.53800000000000003,"15min":0.53800000000000003},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.372,"15min":0.36699999999999999},"min":{"1min":0.247,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.54100000000000004,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.35199999999999998}]},{"osd":3,"last update":"Wed Sep 11 07:58:19 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.41899999999999998,"15min":0.42099999999999999},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.42299999999999999,"15min":0.43099999999999999},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.26700000000000002}]},{"osd":4,"last update":"Wed Sep 11 07:58:40 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53500000000000003,"15min":0.53100000000000003},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52400000000000002,"15min":0.51400000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.68500000000000005}]},{"osd":5,"last update":"Wed Sep 11 07:58:06 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.54200000000000004,"15min":0.54400000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.55500000000000005,"15min":0.56299999999999994},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.64100000000000001,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.57499999999999996}]},{"osd":6,"last update":"Wed Sep 11 07:58:28 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.55300000000000005,"15min":0.55500000000000005},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.68400000000000005,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56399999999999995,"15min":0.56000000000000005},"min":{"1min":0.45200000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.72599999999999998}]},{"osd":7,"last update":"Wed Sep 11 07:57:54 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.71399999999999997}]}]},{"osd":0,"up_from":8,"seq":34359738428,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":298712,"kb_used_data":1356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93442344,"statfs":{"total":95990841344,"available":95684960256,"internally_reserved":0,"allocated":1388544,"data_stored":922697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 07:58:33 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.33900000000000002,"15min":0.30099999999999999},"min":{"1min":0.25600000000000001,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.34000000000000002,"15min":0.308},"min":{"1min":0.255,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.249}]},{"osd":2,"last update":"Wed Sep 11 07:57:54 2024","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.36499999999999999,"15min":0.36099999999999999},"min":{"1min":0.28999999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.496,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.37,"15min":0.36699999999999999},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.47699999999999998,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.38300000000000001}]},{"osd":3,"last update":"Wed Sep 11 07:58:19 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.41999999999999998,"15min":0.42999999999999999},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.54900000000000004,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.42099999999999999,"15min":0.42699999999999999},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.32500000000000001}]},{"osd":4,"last update":"Wed Sep 11 07:58:42 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.504,"15min":0.495},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.52800000000000002,"15min":0.52600000000000002},"min":{"1min":0.39000000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.71999999999999997,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.57899999999999996}]},{"osd":5,"last update":"Wed Sep 11 07:58:02 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.39300000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.55200000000000005,"15min":0.55300000000000005},"min":{"1min":0.374,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.44}]},{"osd":6,"last update":"Wed Sep 11 07:58:27 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55800000000000005,"15min":0.55900000000000005},"min":{"1min":0.39200000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.72499999999999998,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56999999999999995,"15min":0.56899999999999995},"min":{"1min":0.41799999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.73899999999999999,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.51800000000000002}]},{"osd":7,"last update":"Wed Sep 11 07:57:50 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.60199999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T07:58:50.603 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-11T07:58:50.603 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-11T07:58:50.604 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-11T07:58:50.604 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- ceph health --format=json 2024-09-11T07:58:50.748 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:50 smithi139 ceph-mon[27311]: pgmap v34: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:58:50.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:50 smithi139 ceph-mon[27311]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-11T07:58:50.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:50 smithi139 ceph-mon[27311]: Reconfiguring daemon prometheus.a on smithi139 2024-09-11T07:58:50.866 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:58:51.242 INFO:journalctl@ceph.alertmanager.a.smithi110.stdout:Sep 11 07:58:51 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-alertmanager-a[59229]: ts=2024-09-11T07:58:51.212Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000278416s 2024-09-11T07:58:51.528 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:51 smithi110 ceph-mon[25392]: from='client.24650 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:58:51.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:51 smithi110 ceph-mon[31709]: from='client.24650 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:58:51.748 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:51 smithi139 ceph-mon[27311]: from='client.24650 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:58:52.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:52 smithi110 ceph-mon[25392]: pgmap v35: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:58:52.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:52 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:52.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:52 smithi110 ceph-mon[31709]: pgmap v35: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:58:52.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:52 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:52.748 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:52 smithi139 ceph-mon[27311]: pgmap v35: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:58:52.748 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:52 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:53.464 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-11T07:58:53.464 INFO:teuthology.orchestra.run.smithi110.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-11T07:58:53.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:53 smithi110 ceph-mon[25392]: from='client.? 172.21.15.110:0/3868883921' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-11T07:58:53.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:53 smithi110 ceph-mon[31709]: from='client.? 172.21.15.110:0/3868883921' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-11T07:58:53.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:53 smithi139 ceph-mon[27311]: from='client.? 172.21.15.110:0/3868883921' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-11T07:58:54.134 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:58:53 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x[29520]: ::ffff:172.21.15.139 - - [11/Sep/2024:07:58:53] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:58:54.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:54 smithi110 ceph-mon[31709]: pgmap v36: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:58:54.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:54 smithi110 ceph-mon[25392]: pgmap v36: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:58:54.748 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:54 smithi139 ceph-mon[27311]: pgmap v36: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:58:55.349 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-11T07:58:55.349 INFO:tasks.cephadm:Setup complete, yielding 2024-09-11T07:58:55.349 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-11T07:58:55.365 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi110.front.sepia.ceph.com 2024-09-11T07:58:55.367 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- bash -c 'stat -c '"'"'%u %g'"'"' /var/log/ceph | grep '"'"'167 167'"'"'' 2024-09-11T07:58:55.614 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:58:55.994 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:55 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:55.994 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:55 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:55.995 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:55 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:55.995 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:55 smithi110 ceph-mon[25392]: pgmap v37: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:58:55.995 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:55 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:55.995 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:55 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:55.995 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:55 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:55.995 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:55 smithi110 ceph-mon[31709]: pgmap v37: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:58:56.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:55 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:56.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:55 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:56.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:55 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:58:56.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:55 smithi139 ceph-mon[27311]: pgmap v37: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:58:57.385 INFO:teuthology.orchestra.run.smithi110.stdout:167 167 2024-09-11T07:58:58.057 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- bash -c 'ceph orch status' 2024-09-11T07:58:58.095 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:58 smithi139 ceph-mon[27311]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:58:58.095 INFO:journalctl@ceph.iscsi.iscsi.a.smithi139.stdout:Sep 11 07:58:58 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-iscsi-iscsi-a[46755]: debug there is no tcmu-runner data available 2024-09-11T07:58:58.383 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:58 smithi110 ceph-mon[25392]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:58:58.383 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:58 smithi110 ceph-mon[31709]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:58:58.740 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:58:59.161 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:58:59 smithi139 ceph-mon[27311]: from='client.24449 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:58:59.214 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:58:59 smithi110 ceph-mon[25392]: from='client.24449 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:58:59.214 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:58:59 smithi110 ceph-mon[31709]: from='client.24449 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:58:59.468 INFO:journalctl@ceph.alertmanager.a.smithi110.stdout:Sep 11 07:58:59 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-alertmanager-a[59229]: ts=2024-09-11T07:58:59.214Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.002131497s 2024-09-11T07:58:59.748 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:58:59 smithi139 systemd[1]: Stopping Ceph prometheus.a for 41bd1096-7012-11ef-bcea-c7b262605968... 2024-09-11T07:59:00.098 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:59:00 smithi110 ceph-mon[31709]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:59:00.098 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:00 smithi110 ceph-mon[25392]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:59:00.141 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:59:00 smithi139 ceph-mon[27311]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:59:00.417 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:59:00 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a[48872]: ts=2024-09-11T07:59:00.140Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-11T07:59:00.417 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:59:00 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a[48872]: ts=2024-09-11T07:59:00.140Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-09-11T07:59:00.417 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:59:00 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a[48872]: ts=2024-09-11T07:59:00.140Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-09-11T07:59:00.417 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:59:00 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a[48872]: ts=2024-09-11T07:59:00.140Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-11T07:59:00.417 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:59:00 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a[48872]: ts=2024-09-11T07:59:00.140Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-09-11T07:59:00.417 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:59:00 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a[48872]: ts=2024-09-11T07:59:00.140Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-09-11T07:59:00.417 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:59:00 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a[48872]: ts=2024-09-11T07:59:00.141Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-09-11T07:59:00.418 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:59:00 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a[48872]: ts=2024-09-11T07:59:00.141Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-09-11T07:59:00.418 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:59:00 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a[48872]: ts=2024-09-11T07:59:00.141Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-09-11T07:59:00.418 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:59:00 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a[48872]: ts=2024-09-11T07:59:00.141Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-09-11T07:59:00.418 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:59:00 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a[48872]: ts=2024-09-11T07:59:00.141Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-09-11T07:59:00.418 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:59:00 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a[48872]: ts=2024-09-11T07:59:00.141Z caller=main.go:1147 level=info msg="See you next time!" 2024-09-11T07:59:00.418 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:59:00 smithi139 podman[53210]: 2024-09-11 07:59:00.151169542 +0000 UTC m=+0.118118829 container died 5065c2f188985fc64af6e9c6c30a755b18073762ab57a25dbef95a1f019b5c39 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T07:59:00.748 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:59:00 smithi139 podman[53210]: 2024-09-11 07:59:00.565625126 +0000 UTC m=+0.532574402 container cleanup 5065c2f188985fc64af6e9c6c30a755b18073762ab57a25dbef95a1f019b5c39 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T07:59:00.749 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:59:00 smithi139 bash[53210]: ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a 2024-09-11T07:59:01.017 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:59:00 smithi139 podman[53224]: 2024-09-11 07:59:00.766716759 +0000 UTC m=+0.613783053 container remove 5065c2f188985fc64af6e9c6c30a755b18073762ab57a25dbef95a1f019b5c39 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T07:59:01.354 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:59:01 smithi139 systemd[1]: ceph-41bd1096-7012-11ef-bcea-c7b262605968@prometheus.a.service: Deactivated successfully. 2024-09-11T07:59:01.354 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:59:01 smithi139 systemd[1]: Stopped Ceph prometheus.a for 41bd1096-7012-11ef-bcea-c7b262605968. 2024-09-11T07:59:01.354 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:59:01 smithi139 systemd[1]: ceph-41bd1096-7012-11ef-bcea-c7b262605968@prometheus.a.service: Consumed 1.104s CPU time. 2024-09-11T07:59:01.354 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:59:01 smithi139 systemd[1]: Starting Ceph prometheus.a for 41bd1096-7012-11ef-bcea-c7b262605968... 2024-09-11T07:59:01.355 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:59:01 smithi139 podman[53419]: 2024-09-11 07:59:01.139905654 +0000 UTC m=+0.021692895 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-09-11T07:59:01.355 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:59:01 smithi139 podman[53419]: 2024-09-11 07:59:01.251248955 +0000 UTC m=+0.133036188 container create e9f19b573e38df547ffee6d8a3ef058c946c24e9ccdf3a9a3509897545bad3cf (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T07:59:01.703 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:59:01 smithi139 podman[53419]: 2024-09-11 07:59:01.461851536 +0000 UTC m=+0.343638768 container init e9f19b573e38df547ffee6d8a3ef058c946c24e9ccdf3a9a3509897545bad3cf (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T07:59:01.703 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:59:01 smithi139 podman[53419]: 2024-09-11 07:59:01.464539598 +0000 UTC m=+0.346326831 container start e9f19b573e38df547ffee6d8a3ef058c946c24e9ccdf3a9a3509897545bad3cf (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T07:59:01.703 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:59:01 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a[53449]: ts=2024-09-11T07:59:01.488Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-09-11T07:59:01.703 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:59:01 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a[53449]: ts=2024-09-11T07:59:01.488Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-09-11T07:59:01.704 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:59:01 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a[53449]: ts=2024-09-11T07:59:01.488Z caller=main.go:570 level=info host_details="(Linux 5.14.0-505.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Sep 5 07:54:07 UTC 2024 x86_64 smithi139 (none))" 2024-09-11T07:59:01.704 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:59:01 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a[53449]: ts=2024-09-11T07:59:01.488Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-09-11T07:59:01.704 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:59:01 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a[53449]: ts=2024-09-11T07:59:01.488Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-09-11T07:59:01.704 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:59:01 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a[53449]: ts=2024-09-11T07:59:01.489Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-09-11T07:59:01.704 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:59:01 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a[53449]: ts=2024-09-11T07:59:01.490Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-09-11T07:59:01.704 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:59:01 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a[53449]: ts=2024-09-11T07:59:01.491Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-09-11T07:59:01.704 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:59:01 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a[53449]: ts=2024-09-11T07:59:01.491Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-09-11T07:59:01.704 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:59:01 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a[53449]: ts=2024-09-11T07:59:01.493Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-09-11T07:59:01.704 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:59:01 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a[53449]: ts=2024-09-11T07:59:01.493Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.655µs 2024-09-11T07:59:01.704 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:59:01 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a[53449]: ts=2024-09-11T07:59:01.493Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-09-11T07:59:01.704 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:59:01 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a[53449]: ts=2024-09-11T07:59:01.494Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=1 2024-09-11T07:59:01.704 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:59:01 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a[53449]: ts=2024-09-11T07:59:01.494Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=1 2024-09-11T07:59:01.705 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:59:01 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a[53449]: ts=2024-09-11T07:59:01.494Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=19.733µs wal_replay_duration=633.008µs wbl_replay_duration=115ns total_replay_duration=669.67µs 2024-09-11T07:59:01.705 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:59:01 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a[53449]: ts=2024-09-11T07:59:01.495Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-09-11T07:59:01.705 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:59:01 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a[53449]: ts=2024-09-11T07:59:01.495Z caller=main.go:1029 level=info msg="TSDB started" 2024-09-11T07:59:01.705 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:59:01 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a[53449]: ts=2024-09-11T07:59:01.495Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-09-11T07:59:01.705 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:59:01 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a[53449]: ts=2024-09-11T07:59:01.510Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=15.327659ms db_storage=652ns remote_storage=1.103µs web_handler=449ns query_engine=534ns scrape=443.97µs scrape_sd=28.951µs notify=25.571µs notify_sd=8.449µs rules=14.583373ms tracing=6.231µs 2024-09-11T07:59:01.705 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:59:01 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a[53449]: ts=2024-09-11T07:59:01.510Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-09-11T07:59:01.705 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:59:01 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a[53449]: ts=2024-09-11T07:59:01.510Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-09-11T07:59:01.705 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:59:01 smithi139 bash[53419]: e9f19b573e38df547ffee6d8a3ef058c946c24e9ccdf3a9a3509897545bad3cf 2024-09-11T07:59:01.705 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 07:59:01 smithi139 systemd[1]: Started Ceph prometheus.a for 41bd1096-7012-11ef-bcea-c7b262605968. 2024-09-11T07:59:01.837 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:59:01 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: [11/Sep/2024:07:59:01] ENGINE Bus STOPPING 2024-09-11T07:59:01.838 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:01 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:59:01.838 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:01 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:59:01.838 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:01 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:59:01.838 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:01 smithi110 ceph-mon[25392]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:59:01.838 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:01 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:59:01.838 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:01 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:59:01.838 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:01 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-11T07:59:01.838 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:01 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi110.front.sepia.ceph.com:9093"}]: dispatch 2024-09-11T07:59:01.839 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:01 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:59:01.839 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:01 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-11T07:59:01.839 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:01 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi139.front.sepia.ceph.com:9095"}]: dispatch 2024-09-11T07:59:01.839 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:01 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:59:01.839 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:01 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-11T07:59:01.839 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:01 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi139.front.sepia.ceph.com:3000"}]: dispatch 2024-09-11T07:59:01.839 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:01 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:59:01.839 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:01 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:59:01.839 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:01 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:59:01.839 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:01 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:59:01.839 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:01 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:59:01.840 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:59:01 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:59:01.840 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:59:01 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:59:01.840 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:59:01 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:59:01.840 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:59:01 smithi110 ceph-mon[31709]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:59:01.840 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:59:01 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:59:01.840 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:59:01 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:59:01.840 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:59:01 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-11T07:59:01.840 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:59:01 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi110.front.sepia.ceph.com:9093"}]: dispatch 2024-09-11T07:59:01.840 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:59:01 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:59:01.840 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:59:01 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-11T07:59:01.840 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:59:01 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi139.front.sepia.ceph.com:9095"}]: dispatch 2024-09-11T07:59:01.840 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:59:01 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:59:01.841 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:59:01 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-11T07:59:01.841 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:59:01 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi139.front.sepia.ceph.com:3000"}]: dispatch 2024-09-11T07:59:01.841 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:59:01 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:59:01.841 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:59:01 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:59:01.841 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:59:01 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:59:01.841 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:59:01 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:59:01.841 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:59:01 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:59:01.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:59:01 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:59:01.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:59:01 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:59:01.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:59:01 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:59:01.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:59:01 smithi139 ceph-mon[27311]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:59:01.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:59:01 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:59:01.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:59:01 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:59:01.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:59:01 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-11T07:59:01.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:59:01 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi110.front.sepia.ceph.com:9093"}]: dispatch 2024-09-11T07:59:02.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:59:01 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:59:02.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:59:01 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-11T07:59:02.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:59:01 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi139.front.sepia.ceph.com:9095"}]: dispatch 2024-09-11T07:59:02.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:59:01 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:59:02.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:59:01 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-11T07:59:02.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:59:01 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi139.front.sepia.ceph.com:3000"}]: dispatch 2024-09-11T07:59:02.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:59:01 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:59:02.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:59:01 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:59:02.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:59:01 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:59:02.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:59:01 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:59:02.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:59:01 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:59:02.056 INFO:teuthology.orchestra.run.smithi110.stdout:Backend: cephadm 2024-09-11T07:59:02.056 INFO:teuthology.orchestra.run.smithi110.stdout:Available: Yes 2024-09-11T07:59:02.056 INFO:teuthology.orchestra.run.smithi110.stdout:Paused: No 2024-09-11T07:59:02.218 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:59:01 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: [11/Sep/2024:07:59:01] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-11T07:59:02.218 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:59:01 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: [11/Sep/2024:07:59:01] ENGINE Bus STOPPED 2024-09-11T07:59:02.218 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:59:01 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: [11/Sep/2024:07:59:01] ENGINE Bus STARTING 2024-09-11T07:59:02.218 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:59:01 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: [11/Sep/2024:07:59:01] ENGINE Serving on http://:::9283 2024-09-11T07:59:02.218 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:59:01 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: [11/Sep/2024:07:59:01] ENGINE Bus STARTED 2024-09-11T07:59:02.218 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:59:01 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: [11/Sep/2024:07:59:01] ENGINE Bus STOPPING 2024-09-11T07:59:02.819 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:02 smithi110 ceph-mon[25392]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-11T07:59:02.820 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:02 smithi110 ceph-mon[25392]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi110.front.sepia.ceph.com:9093"}]: dispatch 2024-09-11T07:59:02.820 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:02 smithi110 ceph-mon[25392]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-11T07:59:02.820 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:02 smithi110 ceph-mon[25392]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi139.front.sepia.ceph.com:9095"}]: dispatch 2024-09-11T07:59:02.820 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:02 smithi110 ceph-mon[25392]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-11T07:59:02.820 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:02 smithi110 ceph-mon[25392]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi139.front.sepia.ceph.com:3000"}]: dispatch 2024-09-11T07:59:02.820 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:02 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:59:02.820 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:02 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:59:02.820 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:02 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:59:02.820 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:02 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:59:02.820 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:02 smithi110 ceph-mon[25392]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:59:02.820 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:02 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:59:02.821 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:59:02 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: [11/Sep/2024:07:59:02] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-11T07:59:02.821 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:59:02 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: [11/Sep/2024:07:59:02] ENGINE Bus STOPPED 2024-09-11T07:59:02.821 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:59:02 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: [11/Sep/2024:07:59:02] ENGINE Bus STARTING 2024-09-11T07:59:02.821 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:59:02 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: [11/Sep/2024:07:59:02] ENGINE Serving on http://:::9283 2024-09-11T07:59:02.821 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:59:02 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: [11/Sep/2024:07:59:02] ENGINE Bus STARTED 2024-09-11T07:59:02.821 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:59:02 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: [11/Sep/2024:07:59:02] ENGINE Bus STOPPING 2024-09-11T07:59:02.821 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:59:02 smithi110 ceph-mon[31709]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-11T07:59:02.821 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:59:02 smithi110 ceph-mon[31709]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi110.front.sepia.ceph.com:9093"}]: dispatch 2024-09-11T07:59:02.821 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:59:02 smithi110 ceph-mon[31709]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-11T07:59:02.821 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:59:02 smithi110 ceph-mon[31709]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi139.front.sepia.ceph.com:9095"}]: dispatch 2024-09-11T07:59:02.821 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:59:02 smithi110 ceph-mon[31709]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-11T07:59:02.822 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:59:02 smithi110 ceph-mon[31709]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi139.front.sepia.ceph.com:3000"}]: dispatch 2024-09-11T07:59:02.822 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:59:02 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:59:02.822 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:59:02 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:59:02.822 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:59:02 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:59:02.822 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:59:02 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:59:02.822 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:59:02 smithi110 ceph-mon[31709]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:59:02.822 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:59:02 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:59:02.848 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-11T07:59:02.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:59:02 smithi139 ceph-mon[27311]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-11T07:59:02.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:59:02 smithi139 ceph-mon[27311]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi110.front.sepia.ceph.com:9093"}]: dispatch 2024-09-11T07:59:02.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:59:02 smithi139 ceph-mon[27311]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-11T07:59:02.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:59:02 smithi139 ceph-mon[27311]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi139.front.sepia.ceph.com:9095"}]: dispatch 2024-09-11T07:59:02.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:59:02 smithi139 ceph-mon[27311]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-11T07:59:02.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:59:02 smithi139 ceph-mon[27311]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi139.front.sepia.ceph.com:3000"}]: dispatch 2024-09-11T07:59:02.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:59:02 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:59:03.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:59:02 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:59:03.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:59:02 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:59:03.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:59:02 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:59:03.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:59:02 smithi139 ceph-mon[27311]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:59:03.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:59:02 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:59:03.099 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:59:03.355 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:59:03 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: [11/Sep/2024:07:59:03] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-11T07:59:03.355 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:59:03 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: [11/Sep/2024:07:59:03] ENGINE Bus STOPPED 2024-09-11T07:59:03.355 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:59:03 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: [11/Sep/2024:07:59:03] ENGINE Bus STARTING 2024-09-11T07:59:03.355 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:59:03 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: [11/Sep/2024:07:59:03] ENGINE Serving on http://:::9283 2024-09-11T07:59:03.356 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:59:03 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: [11/Sep/2024:07:59:03] ENGINE Bus STARTED 2024-09-11T07:59:03.748 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:59:03 smithi139 ceph-mon[27311]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:59:03.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:59:03 smithi110 ceph-mon[31709]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:59:03.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:03 smithi110 ceph-mon[25392]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:59:04.802 INFO:teuthology.orchestra.run.smithi110.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-11T07:59:04.802 INFO:teuthology.orchestra.run.smithi110.stdout:agent.smithi110 smithi110 running 4s ago 6m - - 2024-09-11T07:59:04.802 INFO:teuthology.orchestra.run.smithi110.stdout:agent.smithi139 smithi139 running 0s ago 6m - - 2024-09-11T07:59:04.802 INFO:teuthology.orchestra.run.smithi110.stdout:alertmanager.a smithi110 *:9093,9094 running (16s) 4s ago 50s 17.4M - 0.25.0 c8568f914cd2 304e26c9eb31 2024-09-11T07:59:04.802 INFO:teuthology.orchestra.run.smithi110.stdout:grafana.a smithi139 *:3000 running (39s) 0s ago 39s 78.5M - 9.4.7 954c08fa6188 551749459dfb 2024-09-11T07:59:04.802 INFO:teuthology.orchestra.run.smithi110.stdout:iscsi.iscsi.a smithi139 *:5000 running (109s) 0s ago 108s 56.7M - 3.8 4a918172a497 0a249abecce9 2024-09-11T07:59:04.802 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.x smithi139 *:8443 running (5m) 0s ago 5m 398M - 17.2.7-1623-gd0e6828a 4a918172a497 61a18e2cfef2 2024-09-11T07:59:04.802 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.y smithi110 *:9283,8765 running (7m) 4s ago 7m 450M - 17.2.7-1623-gd0e6828a 4a918172a497 4cff85987d3d 2024-09-11T07:59:04.802 INFO:teuthology.orchestra.run.smithi110.stdout:mon.a smithi110 running (7m) 4s ago 7m 49.2M 2048M 17.2.7-1623-gd0e6828a 4a918172a497 eb9b8d53622f 2024-09-11T07:59:04.802 INFO:teuthology.orchestra.run.smithi110.stdout:mon.b smithi139 running (5m) 0s ago 5m 42.7M 2048M 17.2.7-1623-gd0e6828a 4a918172a497 4372b1420904 2024-09-11T07:59:04.803 INFO:teuthology.orchestra.run.smithi110.stdout:mon.c smithi110 running (5m) 4s ago 5m 41.2M 2048M 17.2.7-1623-gd0e6828a 4a918172a497 6c2b0e04ed67 2024-09-11T07:59:04.803 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.a smithi110 *:9100 running (32s) 4s ago 31s 3372k - 1.5.0 0da6a335fe13 8c994666edb7 2024-09-11T07:59:04.803 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.b smithi139 *:9100 running (28s) 0s ago 27s 3380k - 1.5.0 0da6a335fe13 82df887f4df0 2024-09-11T07:59:04.803 INFO:teuthology.orchestra.run.smithi110.stdout:osd.0 smithi110 running (5m) 4s ago 5m 45.8M 4096M 17.2.7-1623-gd0e6828a 4a918172a497 f1c060a928c5 2024-09-11T07:59:04.803 INFO:teuthology.orchestra.run.smithi110.stdout:osd.1 smithi110 running (4m) 4s ago 4m 47.8M 4096M 17.2.7-1623-gd0e6828a 4a918172a497 a1259ac22e51 2024-09-11T07:59:04.803 INFO:teuthology.orchestra.run.smithi110.stdout:osd.2 smithi110 running (4m) 4s ago 4m 44.5M 4096M 17.2.7-1623-gd0e6828a 4a918172a497 a98934a65410 2024-09-11T07:59:04.803 INFO:teuthology.orchestra.run.smithi110.stdout:osd.3 smithi110 running (3m) 4s ago 3m 43.3M 4096M 17.2.7-1623-gd0e6828a 4a918172a497 b6bfac7bdfa6 2024-09-11T07:59:04.803 INFO:teuthology.orchestra.run.smithi110.stdout:osd.4 smithi139 running (3m) 0s ago 3m 43.3M 4096M 17.2.7-1623-gd0e6828a 4a918172a497 64fa5b25bebd 2024-09-11T07:59:04.803 INFO:teuthology.orchestra.run.smithi110.stdout:osd.5 smithi139 running (3m) 0s ago 3m 46.0M 4096M 17.2.7-1623-gd0e6828a 4a918172a497 70d5d2bdbd5b 2024-09-11T07:59:04.803 INFO:teuthology.orchestra.run.smithi110.stdout:osd.6 smithi139 running (2m) 0s ago 2m 45.0M 4096M 17.2.7-1623-gd0e6828a 4a918172a497 568d33525e1c 2024-09-11T07:59:04.803 INFO:teuthology.orchestra.run.smithi110.stdout:osd.7 smithi139 running (2m) 0s ago 2m 46.7M 4096M 17.2.7-1623-gd0e6828a 4a918172a497 a6a947f35e55 2024-09-11T07:59:04.803 INFO:teuthology.orchestra.run.smithi110.stdout:prometheus.a smithi139 *:9095 running (83s) 0s ago 83s 28.6M - 2.43.0 a07b618ecd1d 5065c2f18898 2024-09-11T07:59:04.803 INFO:teuthology.orchestra.run.smithi110.stdout:rgw.foo.a smithi110 *:80 running (2m) 4s ago 2m 66.9M - 17.2.7-1623-gd0e6828a 4a918172a497 3cb6994aa1dd 2024-09-11T07:59:05.424 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-11T07:59:05.669 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:59:05.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:05 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:59:05.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:05 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:59:05.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:05 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:59:05.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:05 smithi110 ceph-mon[25392]: from='client.24668 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:59:05.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:05 smithi110 ceph-mon[25392]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:59:05.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:59:05 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:59:05.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:59:05 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:59:05.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:59:05 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:59:05.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:59:05 smithi110 ceph-mon[31709]: from='client.24668 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:59:05.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:59:05 smithi110 ceph-mon[31709]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:59:05.967 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:59:05 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:59:05.967 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:59:05 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:59:05.967 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:59:05 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:59:05.967 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:59:05 smithi139 ceph-mon[27311]: from='client.24668 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:59:05.967 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:59:05 smithi139 ceph-mon[27311]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:59:07.445 INFO:teuthology.orchestra.run.smithi110.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-11T07:59:07.445 INFO:teuthology.orchestra.run.smithi110.stdout:agent 2/2 6s ago 80s * 2024-09-11T07:59:07.445 INFO:teuthology.orchestra.run.smithi110.stdout:alertmanager ?:9093,9094 1/1 6s ago 103s smithi110=a;count:1 2024-09-11T07:59:07.445 INFO:teuthology.orchestra.run.smithi110.stdout:grafana ?:3000 1/1 0s ago 100s smithi139=a;count:1 2024-09-11T07:59:07.445 INFO:teuthology.orchestra.run.smithi110.stdout:iscsi.datapool ?:5000 1/1 0s ago 115s smithi139=iscsi.a;count:1 2024-09-11T07:59:07.445 INFO:teuthology.orchestra.run.smithi110.stdout:mgr 2/2 6s ago 5m smithi110=y;smithi139=x;count:2 2024-09-11T07:59:07.445 INFO:teuthology.orchestra.run.smithi110.stdout:mon 3/3 6s ago 6m smithi110:172.21.15.110=a;smithi110:[v2:172.21.15.110:3301,v1:172.21.15.110:6790]=c;smithi139:172.21.15.139=b;count:3 2024-09-11T07:59:07.445 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter ?:9100 2/2 6s ago 106s smithi110=a;smithi139=b;count:2 2024-09-11T07:59:07.445 INFO:teuthology.orchestra.run.smithi110.stdout:osd 8 6s ago - 2024-09-11T07:59:07.445 INFO:teuthology.orchestra.run.smithi110.stdout:prometheus ?:9095 1/1 0s ago 110s smithi139=a;count:1 2024-09-11T07:59:07.445 INFO:teuthology.orchestra.run.smithi110.stdout:rgw.foo.a ?:80 1/1 6s ago 2m smithi110=foo.a;count:1 2024-09-11T07:59:07.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:59:07 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:59:07.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:59:07 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:59:07.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:59:07 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:59:07.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:59:07 smithi110 ceph-mon[31709]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:59:07.718 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:59:07 smithi110 ceph-mon[31709]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:59:07.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:07 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:59:07.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:07 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:59:07.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:07 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:59:07.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:07 smithi110 ceph-mon[25392]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:59:07.719 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:07 smithi110 ceph-mon[25392]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:59:07.998 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:59:07 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:59:07.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:59:07 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:59:07.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:59:07 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:59:07.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:59:07 smithi139 ceph-mon[27311]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:59:07.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:59:07 smithi139 ceph-mon[27311]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:59:08.198 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-11T07:59:08.441 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:59:08.498 INFO:journalctl@ceph.iscsi.iscsi.a.smithi139.stdout:Sep 11 07:59:08 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-iscsi-iscsi-a[46755]: debug there is no tcmu-runner data available 2024-09-11T07:59:08.867 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:08 smithi110 ceph-mon[25392]: from='client.24449 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:59:08.867 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:59:08 smithi110 ceph-mon[31709]: from='client.24449 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:59:08.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:59:08 smithi139 ceph-mon[27311]: from='client.24449 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:59:09.822 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:59:09 smithi110 ceph-mon[31709]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:59:09.822 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:09 smithi110 ceph-mon[25392]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:59:09.998 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:59:09 smithi139 ceph-mon[27311]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:59:10.299 INFO:teuthology.orchestra.run.smithi110.stdout:HOST ADDR LABELS STATUS 2024-09-11T07:59:10.299 INFO:teuthology.orchestra.run.smithi110.stdout:smithi110 172.21.15.110 2024-09-11T07:59:10.299 INFO:teuthology.orchestra.run.smithi110.stdout:smithi139 172.21.15.139 2024-09-11T07:59:10.299 INFO:teuthology.orchestra.run.smithi110.stdout:2 hosts in cluster 2024-09-11T07:59:10.968 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:10 smithi110 ceph-mon[25392]: from='client.24680 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:59:10.968 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:59:10 smithi110 ceph-mon[31709]: from='client.24680 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:59:10.998 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:59:10 smithi139 ceph-mon[27311]: from='client.24680 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:59:11.004 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-11T07:59:11.247 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:59:11.698 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:11 smithi110 ceph-mon[25392]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:59:11.699 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:59:11 smithi110 ceph-mon[31709]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:59:11.998 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:59:11 smithi139 ceph-mon[27311]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:59:13.081 INFO:teuthology.orchestra.run.smithi110.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-11T07:59:13.081 INFO:teuthology.orchestra.run.smithi110.stdout:smithi110 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7476002T480DGN 447G No 12s ago Has a FileSystem, LVM detected 2024-09-11T07:59:13.081 INFO:teuthology.orchestra.run.smithi110.stdout:smithi110 /dev/nvme1n1 ssd Linux_6156f70d4dbbca17f65d 89.4G No 12s ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-11T07:59:13.081 INFO:teuthology.orchestra.run.smithi110.stdout:smithi110 /dev/nvme2n1 ssd Linux_be3bbd0bcdc85b0f8e9e 89.4G No 12s ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-11T07:59:13.082 INFO:teuthology.orchestra.run.smithi110.stdout:smithi110 /dev/nvme3n1 ssd Linux_de6f9c3c1337e1563f8b 89.4G No 12s ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-11T07:59:13.082 INFO:teuthology.orchestra.run.smithi110.stdout:smithi110 /dev/nvme4n1 ssd Linux_5686d6b55281be7d01c3 89.4G No 12s ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-11T07:59:13.082 INFO:teuthology.orchestra.run.smithi110.stdout:smithi110 /dev/sda hdd ATA_ST1000NM0033-9ZM173_Z1W5QFRD 931G No 12s ago Has partitions 2024-09-11T07:59:13.082 INFO:teuthology.orchestra.run.smithi110.stdout:smithi139 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500N6400BGN 372G No 6s ago Has a FileSystem, LVM detected 2024-09-11T07:59:13.082 INFO:teuthology.orchestra.run.smithi110.stdout:smithi139 /dev/nvme1n1 ssd Linux_6c8cc02e8c293f0e3cb0 89.4G No 6s ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-11T07:59:13.082 INFO:teuthology.orchestra.run.smithi110.stdout:smithi139 /dev/nvme2n1 ssd Linux_635643ae657864394059 89.4G No 6s ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-11T07:59:13.082 INFO:teuthology.orchestra.run.smithi110.stdout:smithi139 /dev/nvme3n1 ssd Linux_0100f88e8ad7605b83c5 89.4G No 6s ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-11T07:59:13.082 INFO:teuthology.orchestra.run.smithi110.stdout:smithi139 /dev/nvme4n1 ssd Linux_1c8c0cd73d601cca786f 89.4G No 6s ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-11T07:59:13.082 INFO:teuthology.orchestra.run.smithi110.stdout:smithi139 /dev/sda hdd ATA_ST1000NM0033-9ZM173_Z1W5XFR4 931G No 6s ago Has partitions 2024-09-11T07:59:13.899 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ls --format yaml' 2024-09-11T07:59:14.026 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:59:13 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x[29520]: ::ffff:172.21.15.139 - - [11/Sep/2024:07:59:13] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:59:14.154 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:14 smithi110 ceph-mon[25392]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:59:14.154 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:14 smithi110 ceph-mon[25392]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:59:14.154 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:59:14 smithi110 ceph-mon[31709]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:59:14.154 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:59:14 smithi110 ceph-mon[31709]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:59:14.496 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:59:14 smithi139 ceph-mon[27311]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:59:14.496 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:59:14 smithi139 ceph-mon[27311]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:59:15.385 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:59:16.051 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:16 smithi110 ceph-mon[25392]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:59:16.052 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:59:15 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: ::ffff:172.21.15.139 - - [11/Sep/2024:07:59:15] "GET /metrics HTTP/1.1" 200 219700 "" "Prometheus/2.43.0" 2024-09-11T07:59:16.052 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:59:16 smithi110 ceph-mon[31709]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:59:16.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:59:16 smithi139 ceph-mon[27311]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:59:17.889 INFO:teuthology.orchestra.run.smithi110.stdout:service_type: agent 2024-09-11T07:59:17.889 INFO:teuthology.orchestra.run.smithi110.stdout:service_name: agent 2024-09-11T07:59:17.889 INFO:teuthology.orchestra.run.smithi110.stdout:placement: 2024-09-11T07:59:17.889 INFO:teuthology.orchestra.run.smithi110.stdout: host_pattern: '*' 2024-09-11T07:59:17.890 INFO:teuthology.orchestra.run.smithi110.stdout:status: 2024-09-11T07:59:17.890 INFO:teuthology.orchestra.run.smithi110.stdout: created: '2024-09-11T07:57:46.977719Z' 2024-09-11T07:59:17.890 INFO:teuthology.orchestra.run.smithi110.stdout: last_refresh: '2024-09-11T07:59:00.696229Z' 2024-09-11T07:59:17.890 INFO:teuthology.orchestra.run.smithi110.stdout: running: 2 2024-09-11T07:59:17.890 INFO:teuthology.orchestra.run.smithi110.stdout: size: 2 2024-09-11T07:59:17.890 INFO:teuthology.orchestra.run.smithi110.stdout:events: 2024-09-11T07:59:17.890 INFO:teuthology.orchestra.run.smithi110.stdout:- 2024-09-11T07:57:48.356283Z service:agent [INFO] "service was created" 2024-09-11T07:59:17.890 INFO:teuthology.orchestra.run.smithi110.stdout:--- 2024-09-11T07:59:17.890 INFO:teuthology.orchestra.run.smithi110.stdout:service_type: alertmanager 2024-09-11T07:59:17.890 INFO:teuthology.orchestra.run.smithi110.stdout:service_name: alertmanager 2024-09-11T07:59:17.890 INFO:teuthology.orchestra.run.smithi110.stdout:placement: 2024-09-11T07:59:17.890 INFO:teuthology.orchestra.run.smithi110.stdout: count: 1 2024-09-11T07:59:17.890 INFO:teuthology.orchestra.run.smithi110.stdout: hosts: 2024-09-11T07:59:17.890 INFO:teuthology.orchestra.run.smithi110.stdout: - smithi110=a 2024-09-11T07:59:17.890 INFO:teuthology.orchestra.run.smithi110.stdout:status: 2024-09-11T07:59:17.891 INFO:teuthology.orchestra.run.smithi110.stdout: created: '2024-09-11T07:57:24.306463Z' 2024-09-11T07:59:17.891 INFO:teuthology.orchestra.run.smithi110.stdout: last_refresh: '2024-09-11T07:59:00.697174Z' 2024-09-11T07:59:17.891 INFO:teuthology.orchestra.run.smithi110.stdout: ports: 2024-09-11T07:59:17.891 INFO:teuthology.orchestra.run.smithi110.stdout: - 9093 2024-09-11T07:59:17.891 INFO:teuthology.orchestra.run.smithi110.stdout: - 9094 2024-09-11T07:59:17.891 INFO:teuthology.orchestra.run.smithi110.stdout: running: 1 2024-09-11T07:59:17.891 INFO:teuthology.orchestra.run.smithi110.stdout: size: 1 2024-09-11T07:59:17.891 INFO:teuthology.orchestra.run.smithi110.stdout:events: 2024-09-11T07:59:17.891 INFO:teuthology.orchestra.run.smithi110.stdout:- 2024-09-11T07:58:14.627939Z service:alertmanager [INFO] "service was created" 2024-09-11T07:59:17.891 INFO:teuthology.orchestra.run.smithi110.stdout:--- 2024-09-11T07:59:17.891 INFO:teuthology.orchestra.run.smithi110.stdout:service_type: grafana 2024-09-11T07:59:17.891 INFO:teuthology.orchestra.run.smithi110.stdout:service_name: grafana 2024-09-11T07:59:17.891 INFO:teuthology.orchestra.run.smithi110.stdout:placement: 2024-09-11T07:59:17.891 INFO:teuthology.orchestra.run.smithi110.stdout: count: 1 2024-09-11T07:59:17.891 INFO:teuthology.orchestra.run.smithi110.stdout: hosts: 2024-09-11T07:59:17.892 INFO:teuthology.orchestra.run.smithi110.stdout: - smithi139=a 2024-09-11T07:59:17.892 INFO:teuthology.orchestra.run.smithi110.stdout:spec: 2024-09-11T07:59:17.892 INFO:teuthology.orchestra.run.smithi110.stdout: anonymous_access: true 2024-09-11T07:59:17.892 INFO:teuthology.orchestra.run.smithi110.stdout:status: 2024-09-11T07:59:17.892 INFO:teuthology.orchestra.run.smithi110.stdout: created: '2024-09-11T07:57:27.221763Z' 2024-09-11T07:59:17.892 INFO:teuthology.orchestra.run.smithi110.stdout: last_refresh: '2024-09-11T07:59:06.524773Z' 2024-09-11T07:59:17.892 INFO:teuthology.orchestra.run.smithi110.stdout: ports: 2024-09-11T07:59:17.892 INFO:teuthology.orchestra.run.smithi110.stdout: - 3000 2024-09-11T07:59:17.892 INFO:teuthology.orchestra.run.smithi110.stdout: running: 1 2024-09-11T07:59:17.892 INFO:teuthology.orchestra.run.smithi110.stdout: size: 1 2024-09-11T07:59:17.892 INFO:teuthology.orchestra.run.smithi110.stdout:events: 2024-09-11T07:59:17.892 INFO:teuthology.orchestra.run.smithi110.stdout:- 2024-09-11T07:58:25.806808Z service:grafana [INFO] "service was created" 2024-09-11T07:59:17.892 INFO:teuthology.orchestra.run.smithi110.stdout:--- 2024-09-11T07:59:17.892 INFO:teuthology.orchestra.run.smithi110.stdout:service_type: iscsi 2024-09-11T07:59:17.893 INFO:teuthology.orchestra.run.smithi110.stdout:service_id: datapool 2024-09-11T07:59:17.893 INFO:teuthology.orchestra.run.smithi110.stdout:service_name: iscsi.datapool 2024-09-11T07:59:17.893 INFO:teuthology.orchestra.run.smithi110.stdout:placement: 2024-09-11T07:59:17.893 INFO:teuthology.orchestra.run.smithi110.stdout: count: 1 2024-09-11T07:59:17.893 INFO:teuthology.orchestra.run.smithi110.stdout: hosts: 2024-09-11T07:59:17.893 INFO:teuthology.orchestra.run.smithi110.stdout: - smithi139=iscsi.a 2024-09-11T07:59:17.893 INFO:teuthology.orchestra.run.smithi110.stdout:spec: 2024-09-11T07:59:17.893 INFO:teuthology.orchestra.run.smithi110.stdout: api_password: admin 2024-09-11T07:59:17.893 INFO:teuthology.orchestra.run.smithi110.stdout: api_port: 5000 2024-09-11T07:59:17.893 INFO:teuthology.orchestra.run.smithi110.stdout: api_user: admin 2024-09-11T07:59:17.893 INFO:teuthology.orchestra.run.smithi110.stdout: pool: datapool 2024-09-11T07:59:17.893 INFO:teuthology.orchestra.run.smithi110.stdout: trusted_ip_list: 172.21.15.139 2024-09-11T07:59:17.893 INFO:teuthology.orchestra.run.smithi110.stdout:status: 2024-09-11T07:59:17.893 INFO:teuthology.orchestra.run.smithi110.stdout: created: '2024-09-11T07:57:12.060319Z' 2024-09-11T07:59:17.893 INFO:teuthology.orchestra.run.smithi110.stdout: last_refresh: '2024-09-11T07:59:06.524650Z' 2024-09-11T07:59:17.894 INFO:teuthology.orchestra.run.smithi110.stdout: ports: 2024-09-11T07:59:17.894 INFO:teuthology.orchestra.run.smithi110.stdout: - 5000 2024-09-11T07:59:17.894 INFO:teuthology.orchestra.run.smithi110.stdout: running: 1 2024-09-11T07:59:17.894 INFO:teuthology.orchestra.run.smithi110.stdout: size: 1 2024-09-11T07:59:17.894 INFO:teuthology.orchestra.run.smithi110.stdout:--- 2024-09-11T07:59:17.894 INFO:teuthology.orchestra.run.smithi110.stdout:service_type: mgr 2024-09-11T07:59:17.894 INFO:teuthology.orchestra.run.smithi110.stdout:service_name: mgr 2024-09-11T07:59:17.894 INFO:teuthology.orchestra.run.smithi110.stdout:placement: 2024-09-11T07:59:17.894 INFO:teuthology.orchestra.run.smithi110.stdout: count: 2 2024-09-11T07:59:17.894 INFO:teuthology.orchestra.run.smithi110.stdout: hosts: 2024-09-11T07:59:17.894 INFO:teuthology.orchestra.run.smithi110.stdout: - smithi110=y 2024-09-11T07:59:17.894 INFO:teuthology.orchestra.run.smithi110.stdout: - smithi139=x 2024-09-11T07:59:17.894 INFO:teuthology.orchestra.run.smithi110.stdout:status: 2024-09-11T07:59:17.894 INFO:teuthology.orchestra.run.smithi110.stdout: created: '2024-09-11T07:53:30.820584Z' 2024-09-11T07:59:17.894 INFO:teuthology.orchestra.run.smithi110.stdout: last_refresh: '2024-09-11T07:59:00.696068Z' 2024-09-11T07:59:17.895 INFO:teuthology.orchestra.run.smithi110.stdout: running: 2 2024-09-11T07:59:17.895 INFO:teuthology.orchestra.run.smithi110.stdout: size: 2 2024-09-11T07:59:17.895 INFO:teuthology.orchestra.run.smithi110.stdout:--- 2024-09-11T07:59:17.895 INFO:teuthology.orchestra.run.smithi110.stdout:service_type: mon 2024-09-11T07:59:17.895 INFO:teuthology.orchestra.run.smithi110.stdout:service_name: mon 2024-09-11T07:59:17.895 INFO:teuthology.orchestra.run.smithi110.stdout:placement: 2024-09-11T07:59:17.895 INFO:teuthology.orchestra.run.smithi110.stdout: count: 3 2024-09-11T07:59:17.895 INFO:teuthology.orchestra.run.smithi110.stdout: hosts: 2024-09-11T07:59:17.895 INFO:teuthology.orchestra.run.smithi110.stdout: - smithi110:172.21.15.110=a 2024-09-11T07:59:17.895 INFO:teuthology.orchestra.run.smithi110.stdout: - smithi110:[v2:172.21.15.110:3301,v1:172.21.15.110:6790]=c 2024-09-11T07:59:17.895 INFO:teuthology.orchestra.run.smithi110.stdout: - smithi139:172.21.15.139=b 2024-09-11T07:59:17.895 INFO:teuthology.orchestra.run.smithi110.stdout:status: 2024-09-11T07:59:17.895 INFO:teuthology.orchestra.run.smithi110.stdout: created: '2024-09-11T07:53:01.016658Z' 2024-09-11T07:59:17.896 INFO:teuthology.orchestra.run.smithi110.stdout: last_refresh: '2024-09-11T07:59:00.695861Z' 2024-09-11T07:59:17.896 INFO:teuthology.orchestra.run.smithi110.stdout: running: 3 2024-09-11T07:59:17.896 INFO:teuthology.orchestra.run.smithi110.stdout: size: 3 2024-09-11T07:59:17.896 INFO:teuthology.orchestra.run.smithi110.stdout:--- 2024-09-11T07:59:17.896 INFO:teuthology.orchestra.run.smithi110.stdout:service_type: node-exporter 2024-09-11T07:59:17.896 INFO:teuthology.orchestra.run.smithi110.stdout:service_name: node-exporter 2024-09-11T07:59:17.896 INFO:teuthology.orchestra.run.smithi110.stdout:placement: 2024-09-11T07:59:17.896 INFO:teuthology.orchestra.run.smithi110.stdout: count: 2 2024-09-11T07:59:17.896 INFO:teuthology.orchestra.run.smithi110.stdout: hosts: 2024-09-11T07:59:17.896 INFO:teuthology.orchestra.run.smithi110.stdout: - smithi110=a 2024-09-11T07:59:17.896 INFO:teuthology.orchestra.run.smithi110.stdout: - smithi139=b 2024-09-11T07:59:17.896 INFO:teuthology.orchestra.run.smithi110.stdout:status: 2024-09-11T07:59:17.896 INFO:teuthology.orchestra.run.smithi110.stdout: created: '2024-09-11T07:57:21.258614Z' 2024-09-11T07:59:17.896 INFO:teuthology.orchestra.run.smithi110.stdout: last_refresh: '2024-09-11T07:59:00.697308Z' 2024-09-11T07:59:17.896 INFO:teuthology.orchestra.run.smithi110.stdout: ports: 2024-09-11T07:59:17.897 INFO:teuthology.orchestra.run.smithi110.stdout: - 9100 2024-09-11T07:59:17.897 INFO:teuthology.orchestra.run.smithi110.stdout: running: 2 2024-09-11T07:59:17.897 INFO:teuthology.orchestra.run.smithi110.stdout: size: 2 2024-09-11T07:59:17.897 INFO:teuthology.orchestra.run.smithi110.stdout:events: 2024-09-11T07:59:17.897 INFO:teuthology.orchestra.run.smithi110.stdout:- 2024-09-11T07:58:37.030683Z service:node-exporter [INFO] "service was created" 2024-09-11T07:59:17.897 INFO:teuthology.orchestra.run.smithi110.stdout:--- 2024-09-11T07:59:17.897 INFO:teuthology.orchestra.run.smithi110.stdout:service_type: osd 2024-09-11T07:59:17.897 INFO:teuthology.orchestra.run.smithi110.stdout:service_name: osd 2024-09-11T07:59:17.897 INFO:teuthology.orchestra.run.smithi110.stdout:unmanaged: true 2024-09-11T07:59:17.897 INFO:teuthology.orchestra.run.smithi110.stdout:spec: 2024-09-11T07:59:17.897 INFO:teuthology.orchestra.run.smithi110.stdout: filter_logic: AND 2024-09-11T07:59:17.897 INFO:teuthology.orchestra.run.smithi110.stdout: objectstore: bluestore 2024-09-11T07:59:17.897 INFO:teuthology.orchestra.run.smithi110.stdout:status: 2024-09-11T07:59:17.897 INFO:teuthology.orchestra.run.smithi110.stdout: container_image_id: 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b 2024-09-11T07:59:17.898 INFO:teuthology.orchestra.run.smithi110.stdout: container_image_name: quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T07:59:17.898 INFO:teuthology.orchestra.run.smithi110.stdout: last_refresh: '2024-09-11T07:59:00.696449Z' 2024-09-11T07:59:17.898 INFO:teuthology.orchestra.run.smithi110.stdout: running: 8 2024-09-11T07:59:17.898 INFO:teuthology.orchestra.run.smithi110.stdout: size: 8 2024-09-11T07:59:17.898 INFO:teuthology.orchestra.run.smithi110.stdout:--- 2024-09-11T07:59:17.898 INFO:teuthology.orchestra.run.smithi110.stdout:service_type: prometheus 2024-09-11T07:59:17.898 INFO:teuthology.orchestra.run.smithi110.stdout:service_name: prometheus 2024-09-11T07:59:17.898 INFO:teuthology.orchestra.run.smithi110.stdout:placement: 2024-09-11T07:59:17.898 INFO:teuthology.orchestra.run.smithi110.stdout: count: 1 2024-09-11T07:59:17.898 INFO:teuthology.orchestra.run.smithi110.stdout: hosts: 2024-09-11T07:59:17.898 INFO:teuthology.orchestra.run.smithi110.stdout: - smithi139=a 2024-09-11T07:59:17.898 INFO:teuthology.orchestra.run.smithi110.stdout:status: 2024-09-11T07:59:17.898 INFO:teuthology.orchestra.run.smithi110.stdout: created: '2024-09-11T07:57:16.798939Z' 2024-09-11T07:59:17.899 INFO:teuthology.orchestra.run.smithi110.stdout: last_refresh: '2024-09-11T07:59:06.524711Z' 2024-09-11T07:59:17.899 INFO:teuthology.orchestra.run.smithi110.stdout: ports: 2024-09-11T07:59:17.899 INFO:teuthology.orchestra.run.smithi110.stdout: - 9095 2024-09-11T07:59:17.899 INFO:teuthology.orchestra.run.smithi110.stdout: running: 1 2024-09-11T07:59:17.899 INFO:teuthology.orchestra.run.smithi110.stdout: size: 1 2024-09-11T07:59:17.899 INFO:teuthology.orchestra.run.smithi110.stdout:events: 2024-09-11T07:59:17.899 INFO:teuthology.orchestra.run.smithi110.stdout:- 2024-09-11T07:58:37.036557Z service:prometheus [INFO] "service was created" 2024-09-11T07:59:17.899 INFO:teuthology.orchestra.run.smithi110.stdout:--- 2024-09-11T07:59:17.899 INFO:teuthology.orchestra.run.smithi110.stdout:service_type: rgw 2024-09-11T07:59:17.899 INFO:teuthology.orchestra.run.smithi110.stdout:service_id: foo.a 2024-09-11T07:59:17.899 INFO:teuthology.orchestra.run.smithi110.stdout:service_name: rgw.foo.a 2024-09-11T07:59:17.899 INFO:teuthology.orchestra.run.smithi110.stdout:placement: 2024-09-11T07:59:17.899 INFO:teuthology.orchestra.run.smithi110.stdout: count: 1 2024-09-11T07:59:17.900 INFO:teuthology.orchestra.run.smithi110.stdout: hosts: 2024-09-11T07:59:17.900 INFO:teuthology.orchestra.run.smithi110.stdout: - smithi110=foo.a 2024-09-11T07:59:17.900 INFO:teuthology.orchestra.run.smithi110.stdout:status: 2024-09-11T07:59:17.900 INFO:teuthology.orchestra.run.smithi110.stdout: created: '2024-09-11T07:57:03.004376Z' 2024-09-11T07:59:17.900 INFO:teuthology.orchestra.run.smithi110.stdout: last_refresh: '2024-09-11T07:59:00.697021Z' 2024-09-11T07:59:17.900 INFO:teuthology.orchestra.run.smithi110.stdout: ports: 2024-09-11T07:59:17.900 INFO:teuthology.orchestra.run.smithi110.stdout: - 80 2024-09-11T07:59:17.900 INFO:teuthology.orchestra.run.smithi110.stdout: running: 1 2024-09-11T07:59:17.900 INFO:teuthology.orchestra.run.smithi110.stdout: size: 1 2024-09-11T07:59:18.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:18 smithi110 ceph-mon[25392]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:59:18.218 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:18 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:59:18.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:59:18 smithi110 ceph-mon[31709]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:59:18.219 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:59:18 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:59:18.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:59:18 smithi139 ceph-mon[27311]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:59:18.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:59:18 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:59:18.499 INFO:journalctl@ceph.iscsi.iscsi.a.smithi139.stdout:Sep 11 07:59:18 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-iscsi-iscsi-a[46755]: debug there is no tcmu-runner data available 2024-09-11T07:59:18.781 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 41bd1096-7012-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ls | grep '"'"'^osd '"'"'' 2024-09-11T07:59:19.025 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:59:19.287 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:59:19 smithi110 ceph-mon[31709]: from='client.24692 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "yaml"}]: dispatch 2024-09-11T07:59:19.287 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:59:19 smithi110 ceph-mon[31709]: from='client.24449 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:59:19.287 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:19 smithi110 ceph-mon[25392]: from='client.24692 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "yaml"}]: dispatch 2024-09-11T07:59:19.287 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:19 smithi110 ceph-mon[25392]: from='client.24449 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:59:19.498 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:59:19 smithi139 ceph-mon[27311]: from='client.24692 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "yaml"}]: dispatch 2024-09-11T07:59:19.499 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:59:19 smithi139 ceph-mon[27311]: from='client.24449 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:59:20.249 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:59:20 smithi139 ceph-mon[27311]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:59:20.346 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:20 smithi110 ceph-mon[25392]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:59:20.346 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:59:20 smithi110 ceph-mon[31709]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:59:20.760 INFO:teuthology.orchestra.run.smithi110.stdout:osd 8 20s ago - 2024-09-11T07:59:21.480 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-11T07:59:21.488 INFO:tasks.cephadm:Teardown begin 2024-09-11T07:59:21.488 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T07:59:21.520 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T07:59:21.551 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-11T07:59:21.551 DEBUG:teuthology.orchestra.run.smithi110:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-11T07:59:21.576 DEBUG:teuthology.orchestra.run.smithi139:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-11T07:59:21.606 INFO:tasks.cephadm:Stopping all daemons... 2024-09-11T07:59:21.606 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-11T07:59:21.606 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-41bd1096-7012-11ef-bcea-c7b262605968@mon.a 2024-09-11T07:59:21.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:21 smithi110 systemd[1]: Stopping Ceph mon.a for 41bd1096-7012-11ef-bcea-c7b262605968... 2024-09-11T07:59:22.134 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:59:21 smithi139 ceph-mon[27311]: from='client.24698 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:59:22.134 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:59:21 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:59:22.134 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:59:21 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:59:22.134 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:59:21 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:59:22.134 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:59:21 smithi139 ceph-mon[27311]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:59:22.135 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:59:21 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:59:22.135 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:59:21 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:59:22.135 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:59:21 smithi139 ceph-mon[27311]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:59:22.218 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:59:21 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: [11/Sep/2024:07:59:21] ENGINE Bus STOPPING 2024-09-11T07:59:22.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:59:21 smithi110 ceph-mon[31709]: from='client.24698 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:59:22.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:59:21 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:59:22.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:59:21 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:59:22.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:59:21 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:59:22.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:59:21 smithi110 ceph-mon[31709]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:59:22.219 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:59:21 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:59:22.219 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:59:21 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:59:22.219 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:59:21 smithi110 ceph-mon[31709]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:59:22.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:21 smithi110 ceph-mon[25392]: from='client.24698 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:59:22.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:21 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:59:22.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:21 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:59:22.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:21 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:59:22.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:21 smithi110 ceph-mon[25392]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:59:22.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:21 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:59:22.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:21 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:59:22.219 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:21 smithi110 ceph-mon[25392]: from='mgr.24500 172.21.15.110:0/4289037701' entity='mgr.y' 2024-09-11T07:59:22.220 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:21 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mon-a[25388]: 2024-09-11T07:59:21.925+0000 7f491c9bf640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T07:59:22.220 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:21 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mon-a[25388]: 2024-09-11T07:59:21.925+0000 7f491c9bf640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-11T07:59:22.220 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:22 smithi110 podman[61800]: 2024-09-11 07:59:22.06989131 +0000 UTC m=+0.271695436 container died eb9b8d53622fef5a3fddafcb96b4053cf087eee1fa8873ff87dbf3ecbc2aafae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-mon-a, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-11T07:59:22.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:22 smithi110 podman[61800]: 2024-09-11 07:59:22.579476631 +0000 UTC m=+0.781280748 container cleanup eb9b8d53622fef5a3fddafcb96b4053cf087eee1fa8873ff87dbf3ecbc2aafae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-mon-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2) 2024-09-11T07:59:22.718 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:22 smithi110 bash[61800]: ceph-41bd1096-7012-11ef-bcea-c7b262605968-mon-a 2024-09-11T07:59:22.718 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:59:22 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: [11/Sep/2024:07:59:22] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-11T07:59:22.718 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:59:22 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: [11/Sep/2024:07:59:22] ENGINE Bus STOPPED 2024-09-11T07:59:22.719 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:59:22 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: [11/Sep/2024:07:59:22] ENGINE Bus STARTING 2024-09-11T07:59:22.719 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:59:22 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: [11/Sep/2024:07:59:22] ENGINE Serving on http://:::9283 2024-09-11T07:59:22.719 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:59:22 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: [11/Sep/2024:07:59:22] ENGINE Bus STARTED 2024-09-11T07:59:23.057 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-41bd1096-7012-11ef-bcea-c7b262605968@mon.a.service' 2024-09-11T07:59:23.079 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:22 smithi110 podman[61812]: 2024-09-11 07:59:22.815694352 +0000 UTC m=+0.748111242 container remove eb9b8d53622fef5a3fddafcb96b4053cf087eee1fa8873ff87dbf3ecbc2aafae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T07:59:23.079 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:23 smithi110 systemd[1]: ceph-41bd1096-7012-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-11T07:59:23.079 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:23 smithi110 systemd[1]: Stopped Ceph mon.a for 41bd1096-7012-11ef-bcea-c7b262605968. 2024-09-11T07:59:23.079 INFO:journalctl@ceph.mon.a.smithi110.stdout:Sep 11 07:59:23 smithi110 systemd[1]: ceph-41bd1096-7012-11ef-bcea-c7b262605968@mon.a.service: Consumed 10.412s CPU time. 2024-09-11T07:59:23.108 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T07:59:23.108 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-11T07:59:23.109 INFO:tasks.cephadm.mon.b:Stopping mon.c... 2024-09-11T07:59:23.109 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-41bd1096-7012-11ef-bcea-c7b262605968@mon.c 2024-09-11T07:59:23.419 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:59:23 smithi110 systemd[1]: Stopping Ceph mon.c for 41bd1096-7012-11ef-bcea-c7b262605968... 2024-09-11T07:59:23.718 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:59:23 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: [11/Sep/2024:07:59:23] ENGINE Bus STOPPING 2024-09-11T07:59:23.718 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:59:23 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: [11/Sep/2024:07:59:23] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-11T07:59:23.718 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:59:23 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: [11/Sep/2024:07:59:23] ENGINE Bus STOPPED 2024-09-11T07:59:23.719 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:59:23 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: [11/Sep/2024:07:59:23] ENGINE Bus STARTING 2024-09-11T07:59:23.719 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:59:23 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: [11/Sep/2024:07:59:23] ENGINE Serving on http://:::9283 2024-09-11T07:59:23.719 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:59:23 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: [11/Sep/2024:07:59:23] ENGINE Bus STARTED 2024-09-11T07:59:23.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:59:23 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mon-c[31705]: 2024-09-11T07:59:23.417+0000 7fd8027a7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T07:59:23.719 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:59:23 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mon-c[31705]: 2024-09-11T07:59:23.417+0000 7fd8027a7640 -1 mon.c@2(peon) e3 *** Got Signal Terminated *** 2024-09-11T07:59:23.720 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:59:23 smithi110 podman[61934]: 2024-09-11 07:59:23.633378921 +0000 UTC m=+0.310370010 container died 6c2b0e04ed670ee58c9b649addec5719a6067843abb9f26435d8f00388bbb113 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T07:59:24.249 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:59:23 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x[29520]: ::ffff:172.21.15.139 - - [11/Sep/2024:07:59:23] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:59:24.468 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:59:24 smithi110 podman[61934]: 2024-09-11 07:59:24.196332657 +0000 UTC m=+0.873323736 container cleanup 6c2b0e04ed670ee58c9b649addec5719a6067843abb9f26435d8f00388bbb113 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-mon-c, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-09-11T07:59:24.468 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 11 07:59:24 smithi110 bash[61934]: ceph-41bd1096-7012-11ef-bcea-c7b262605968-mon-c 2024-09-11T07:59:24.665 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-41bd1096-7012-11ef-bcea-c7b262605968@mon.c.service' 2024-09-11T07:59:24.700 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T07:59:24.701 INFO:tasks.cephadm.mon.b:Stopped mon.c 2024-09-11T07:59:24.701 INFO:tasks.cephadm.mon.b:Stopping mon.b... 2024-09-11T07:59:24.701 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ceph-41bd1096-7012-11ef-bcea-c7b262605968@mon.b 2024-09-11T07:59:24.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:59:24 smithi139 systemd[1]: Stopping Ceph mon.b for 41bd1096-7012-11ef-bcea-c7b262605968... 2024-09-11T07:59:25.404 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:59:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mon-b[27298]: 2024-09-11T07:59:25.001+0000 7fb558571640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T07:59:25.404 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:59:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mon-b[27298]: 2024-09-11T07:59:25.001+0000 7fb558571640 -1 mon.b@1(peon) e3 *** Got Signal Terminated *** 2024-09-11T07:59:25.404 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:59:25 smithi139 podman[54449]: 2024-09-11 07:59:25.125285217 +0000 UTC m=+0.237578940 container died 4372b142090407d319454577968ec4111fd04b600ecedf8fd0bdf2cc2aec6f80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-mon-b, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T07:59:25.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:59:25 smithi139 podman[54449]: 2024-09-11 07:59:25.50250012 +0000 UTC m=+0.614793831 container cleanup 4372b142090407d319454577968ec4111fd04b600ecedf8fd0bdf2cc2aec6f80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-mon-b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD) 2024-09-11T07:59:25.749 INFO:journalctl@ceph.mon.b.smithi139.stdout:Sep 11 07:59:25 smithi139 bash[54449]: ceph-41bd1096-7012-11ef-bcea-c7b262605968-mon-b 2024-09-11T07:59:25.932 DEBUG:teuthology.orchestra.run.smithi139:> sudo pkill -f 'journalctl -f -n 0 -u ceph-41bd1096-7012-11ef-bcea-c7b262605968@mon.b.service' 2024-09-11T07:59:25.978 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T07:59:25.979 INFO:tasks.cephadm.mon.b:Stopped mon.b 2024-09-11T07:59:25.979 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-11T07:59:25.979 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-41bd1096-7012-11ef-bcea-c7b262605968@mgr.y 2024-09-11T07:59:26.217 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:59:25 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: ::ffff:172.21.15.139 - - [11/Sep/2024:07:59:25] "GET /metrics HTTP/1.1" 200 219702 "" "Prometheus/2.43.0" 2024-09-11T07:59:26.218 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:59:26 smithi110 systemd[1]: Stopping Ceph mgr.y for 41bd1096-7012-11ef-bcea-c7b262605968... 2024-09-11T07:59:26.645 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:59:26 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:59:26.291+0000 7f86550bc640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.y -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T07:59:26.645 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:59:26 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y[25618]: 2024-09-11T07:59:26.291+0000 7f86550bc640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-11T07:59:26.645 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:59:26 smithi110 podman[62070]: 2024-09-11 07:59:26.323933948 +0000 UTC m=+0.144975655 container died 4cff85987d3da685fed26e768114f2342f49b7dae8329397ade02299eedf8bd0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-11T07:59:26.968 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:59:26 smithi110 podman[62070]: 2024-09-11 07:59:26.785072348 +0000 UTC m=+0.606114054 container cleanup 4cff85987d3da685fed26e768114f2342f49b7dae8329397ade02299eedf8bd0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T07:59:26.968 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Sep 11 07:59:26 smithi110 bash[62070]: ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-y 2024-09-11T07:59:27.208 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-41bd1096-7012-11ef-bcea-c7b262605968@mgr.y.service' 2024-09-11T07:59:27.243 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T07:59:27.243 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-11T07:59:27.243 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-11T07:59:27.243 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ceph-41bd1096-7012-11ef-bcea-c7b262605968@mgr.x 2024-09-11T07:59:27.546 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:59:27 smithi139 systemd[1]: Stopping Ceph mgr.x for 41bd1096-7012-11ef-bcea-c7b262605968... 2024-09-11T07:59:27.800 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:59:27 smithi139 podman[54574]: 2024-09-11 07:59:27.544971173 +0000 UTC m=+0.108288127 container died 61a18e2cfef24f75da2294ba4d3b4c7eaa37089343ed1e3e09b9b37787eb1a60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T07:59:28.067 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:59:27 smithi139 podman[54574]: 2024-09-11 07:59:27.890671322 +0000 UTC m=+0.453988270 container cleanup 61a18e2cfef24f75da2294ba4d3b4c7eaa37089343ed1e3e09b9b37787eb1a60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T07:59:28.067 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:59:27 smithi139 bash[54574]: ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x 2024-09-11T07:59:28.304 DEBUG:teuthology.orchestra.run.smithi139:> sudo pkill -f 'journalctl -f -n 0 -u ceph-41bd1096-7012-11ef-bcea-c7b262605968@mgr.x.service' 2024-09-11T07:59:28.325 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:59:28 smithi139 podman[54588]: 2024-09-11 07:59:28.066858987 +0000 UTC m=+0.512239872 container remove 61a18e2cfef24f75da2294ba4d3b4c7eaa37089343ed1e3e09b9b37787eb1a60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS) 2024-09-11T07:59:28.325 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:59:28 smithi139 systemd[1]: ceph-41bd1096-7012-11ef-bcea-c7b262605968@mgr.x.service: Main process exited, code=exited, status=143/n/a 2024-09-11T07:59:28.325 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:59:28 smithi139 systemd[1]: ceph-41bd1096-7012-11ef-bcea-c7b262605968@mgr.x.service: Failed with result 'exit-code'. 2024-09-11T07:59:28.325 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:59:28 smithi139 systemd[1]: Stopped Ceph mgr.x for 41bd1096-7012-11ef-bcea-c7b262605968. 2024-09-11T07:59:28.325 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 11 07:59:28 smithi139 systemd[1]: ceph-41bd1096-7012-11ef-bcea-c7b262605968@mgr.x.service: Consumed 10.267s CPU time. 2024-09-11T07:59:28.338 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T07:59:28.338 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-11T07:59:28.338 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-11T07:59:28.339 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-41bd1096-7012-11ef-bcea-c7b262605968@osd.0 2024-09-11T07:59:28.650 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 11 07:59:28 smithi110 systemd[1]: Stopping Ceph osd.0 for 41bd1096-7012-11ef-bcea-c7b262605968... 2024-09-11T07:59:28.968 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 11 07:59:28 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-0[36077]: 2024-09-11T07:59:28.648+0000 7f70e9a76640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T07:59:28.968 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 11 07:59:28 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-0[36077]: 2024-09-11T07:59:28.648+0000 7f70e9a76640 -1 osd.0 71 *** Got signal Terminated *** 2024-09-11T07:59:28.968 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 11 07:59:28 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-0[36077]: 2024-09-11T07:59:28.648+0000 7f70e9a76640 -1 osd.0 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T07:59:34.443 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 11 07:59:34 smithi110 podman[62192]: 2024-09-11 07:59:34.177221862 +0000 UTC m=+5.649842404 container died f1c060a928c55cd246f256039e48312f20ee5aaad15f2e15f35ebb8881726d0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2) 2024-09-11T07:59:34.710 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 11 07:59:34 smithi110 podman[62192]: 2024-09-11 07:59:34.516245594 +0000 UTC m=+5.988866136 container cleanup f1c060a928c55cd246f256039e48312f20ee5aaad15f2e15f35ebb8881726d0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-11T07:59:34.710 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 11 07:59:34 smithi110 bash[62192]: ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-0 2024-09-11T07:59:34.967 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 11 07:59:34 smithi110 podman[62361]: 2024-09-11 07:59:34.70901084 +0000 UTC m=+0.532452464 container remove f1c060a928c55cd246f256039e48312f20ee5aaad15f2e15f35ebb8881726d0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-11T07:59:35.378 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 11 07:59:35 smithi110 podman[62449]: 2024-09-11 07:59:35.028818777 +0000 UTC m=+0.015666970 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T07:59:35.378 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 11 07:59:35 smithi110 podman[62449]: 2024-09-11 07:59:35.276433972 +0000 UTC m=+0.263282171 container create c259eb26923d5c2af8ca399a064d83801cf64d9cd182f44fd1b75d5b3c32adb3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True) 2024-09-11T07:59:35.710 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 11 07:59:35 smithi110 podman[62449]: 2024-09-11 07:59:35.521624497 +0000 UTC m=+0.508472691 container init c259eb26923d5c2af8ca399a064d83801cf64d9cd182f44fd1b75d5b3c32adb3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T07:59:35.710 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 11 07:59:35 smithi110 podman[62449]: 2024-09-11 07:59:35.52513174 +0000 UTC m=+0.511979935 container start c259eb26923d5c2af8ca399a064d83801cf64d9cd182f44fd1b75d5b3c32adb3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-11T07:59:35.710 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 11 07:59:35 smithi110 podman[62449]: 2024-09-11 07:59:35.621929206 +0000 UTC m=+0.608777416 container attach c259eb26923d5c2af8ca399a064d83801cf64d9cd182f44fd1b75d5b3c32adb3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-0-deactivate, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True) 2024-09-11T07:59:35.968 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 11 07:59:35 smithi110 podman[62449]: 2024-09-11 07:59:35.709033617 +0000 UTC m=+0.695881822 container died c259eb26923d5c2af8ca399a064d83801cf64d9cd182f44fd1b75d5b3c32adb3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T07:59:36.399 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-41bd1096-7012-11ef-bcea-c7b262605968@osd.0.service' 2024-09-11T07:59:36.441 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T07:59:36.441 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-11T07:59:36.441 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-11T07:59:36.441 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-41bd1096-7012-11ef-bcea-c7b262605968@osd.1 2024-09-11T07:59:36.718 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 07:59:36 smithi110 systemd[1]: Stopping Ceph osd.1 for 41bd1096-7012-11ef-bcea-c7b262605968... 2024-09-11T07:59:37.218 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 07:59:36 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-1[39572]: 2024-09-11T07:59:36.789+0000 7fdf4ac71640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T07:59:37.218 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 07:59:36 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-1[39572]: 2024-09-11T07:59:36.789+0000 7fdf4ac71640 -1 osd.1 71 *** Got signal Terminated *** 2024-09-11T07:59:37.218 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 07:59:36 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-1[39572]: 2024-09-11T07:59:36.789+0000 7fdf4ac71640 -1 osd.1 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T07:59:42.468 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 07:59:42 smithi110 podman[62551]: 2024-09-11 07:59:42.194783382 +0000 UTC m=+5.532303190 container died a1259ac22e51af316f0f902c1095e881649acc093096cb4ec5340ebd881cc443 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-1, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-11T07:59:42.909 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 07:59:42 smithi110 podman[62551]: 2024-09-11 07:59:42.665943691 +0000 UTC m=+6.003463532 container cleanup a1259ac22e51af316f0f902c1095e881649acc093096cb4ec5340ebd881cc443 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-1, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T07:59:42.909 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 07:59:42 smithi110 bash[62551]: ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-1 2024-09-11T07:59:43.217 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 07:59:42 smithi110 podman[62727]: 2024-09-11 07:59:42.908822592 +0000 UTC m=+0.709011012 container remove a1259ac22e51af316f0f902c1095e881649acc093096cb4ec5340ebd881cc443 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-1, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T07:59:43.513 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 07:59:43 smithi110 podman[62849]: 2024-09-11 07:59:43.138262349 +0000 UTC m=+0.017598015 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T07:59:43.513 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 07:59:43 smithi110 podman[62849]: 2024-09-11 07:59:43.27672989 +0000 UTC m=+0.156065588 container create fde00f64f9aee68839f88833ed3f161ea0ba12b32166e74c2c0c080774c0c88e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-11T07:59:43.968 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 07:59:43 smithi110 podman[62849]: 2024-09-11 07:59:43.512243523 +0000 UTC m=+0.391579165 container init fde00f64f9aee68839f88833ed3f161ea0ba12b32166e74c2c0c080774c0c88e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=) 2024-09-11T07:59:43.968 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 07:59:43 smithi110 podman[62849]: 2024-09-11 07:59:43.515598213 +0000 UTC m=+0.394933856 container start fde00f64f9aee68839f88833ed3f161ea0ba12b32166e74c2c0c080774c0c88e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-11T07:59:43.968 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 07:59:43 smithi110 podman[62849]: 2024-09-11 07:59:43.595400655 +0000 UTC m=+0.474736314 container attach fde00f64f9aee68839f88833ed3f161ea0ba12b32166e74c2c0c080774c0c88e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=) 2024-09-11T07:59:43.968 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 11 07:59:43 smithi110 podman[62849]: 2024-09-11 07:59:43.694867094 +0000 UTC m=+0.574202746 container died fde00f64f9aee68839f88833ed3f161ea0ba12b32166e74c2c0c080774c0c88e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-1-deactivate, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-11T07:59:44.366 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-41bd1096-7012-11ef-bcea-c7b262605968@osd.1.service' 2024-09-11T07:59:44.406 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T07:59:44.406 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-11T07:59:44.406 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-11T07:59:44.406 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-41bd1096-7012-11ef-bcea-c7b262605968@osd.2 2024-09-11T07:59:44.718 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 11 07:59:44 smithi110 systemd[1]: Stopping Ceph osd.2 for 41bd1096-7012-11ef-bcea-c7b262605968... 2024-09-11T07:59:45.218 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 11 07:59:44 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-2[43117]: 2024-09-11T07:59:44.753+0000 7f459e081640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T07:59:45.218 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 11 07:59:44 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-2[43117]: 2024-09-11T07:59:44.753+0000 7f459e081640 -1 osd.2 71 *** Got signal Terminated *** 2024-09-11T07:59:45.218 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 11 07:59:44 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-2[43117]: 2024-09-11T07:59:44.753+0000 7f459e081640 -1 osd.2 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T07:59:50.839 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 11 07:59:50 smithi110 podman[63001]: 2024-09-11 07:59:50.55046691 +0000 UTC m=+5.914629334 container died a98934a654109bd8af41c0a5978de473c6fad814206a2e486faa2c7fd7e77d07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T07:59:51.130 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 11 07:59:50 smithi110 podman[63001]: 2024-09-11 07:59:50.953513009 +0000 UTC m=+6.317675420 container cleanup a98934a654109bd8af41c0a5978de473c6fad814206a2e486faa2c7fd7e77d07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T07:59:51.130 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 11 07:59:50 smithi110 bash[63001]: ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-2 2024-09-11T07:59:51.390 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 11 07:59:51 smithi110 podman[63404]: 2024-09-11 07:59:51.129971842 +0000 UTC m=+0.905609586 container remove a98934a654109bd8af41c0a5978de473c6fad814206a2e486faa2c7fd7e77d07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-11T07:59:52.145 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 11 07:59:51 smithi110 podman[63533]: 2024-09-11 07:59:51.794032461 +0000 UTC m=+0.016347449 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T07:59:52.145 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 11 07:59:51 smithi110 podman[63533]: 2024-09-11 07:59:51.911888893 +0000 UTC m=+0.134203863 container create e7d57fea5756328c355bc18914164430505a7a1f5a568beb09a9725756166527 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-11T07:59:52.145 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 11 07:59:52 smithi110 podman[63533]: 2024-09-11 07:59:52.144268532 +0000 UTC m=+0.366583502 container init e7d57fea5756328c355bc18914164430505a7a1f5a568beb09a9725756166527 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-2-deactivate, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T07:59:52.405 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 11 07:59:52 smithi110 podman[63533]: 2024-09-11 07:59:52.147788624 +0000 UTC m=+0.370103594 container start e7d57fea5756328c355bc18914164430505a7a1f5a568beb09a9725756166527 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-11T07:59:52.406 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 11 07:59:52 smithi110 podman[63533]: 2024-09-11 07:59:52.236195534 +0000 UTC m=+0.458510526 container attach e7d57fea5756328c355bc18914164430505a7a1f5a568beb09a9725756166527 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-11T07:59:52.406 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 11 07:59:52 smithi110 podman[63533]: 2024-09-11 07:59:52.405175027 +0000 UTC m=+0.627490007 container died e7d57fea5756328c355bc18914164430505a7a1f5a568beb09a9725756166527 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-11T07:59:53.439 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-41bd1096-7012-11ef-bcea-c7b262605968@osd.2.service' 2024-09-11T07:59:53.482 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T07:59:53.482 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-11T07:59:53.482 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-11T07:59:53.483 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-41bd1096-7012-11ef-bcea-c7b262605968@osd.3 2024-09-11T07:59:53.718 INFO:journalctl@ceph.osd.3.smithi110.stdout:Sep 11 07:59:53 smithi110 systemd[1]: Stopping Ceph osd.3 for 41bd1096-7012-11ef-bcea-c7b262605968... 2024-09-11T07:59:54.218 INFO:journalctl@ceph.osd.3.smithi110.stdout:Sep 11 07:59:53 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-3[46734]: 2024-09-11T07:59:53.812+0000 7f1dbc488640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T07:59:54.218 INFO:journalctl@ceph.osd.3.smithi110.stdout:Sep 11 07:59:53 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-3[46734]: 2024-09-11T07:59:53.812+0000 7f1dbc488640 -1 osd.3 71 *** Got signal Terminated *** 2024-09-11T07:59:54.218 INFO:journalctl@ceph.osd.3.smithi110.stdout:Sep 11 07:59:53 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-3[46734]: 2024-09-11T07:59:53.812+0000 7f1dbc488640 -1 osd.3 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T07:59:55.492 INFO:journalctl@ceph.osd.4.smithi139.stdout:Sep 11 07:59:55 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-4[33592]: 2024-09-11T07:59:55.029+0000 7fd5a86f5640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:30.198491+0000 front 2024-09-11T07:59:30.198479+0000 (oldest deadline 2024-09-11T07:59:54.298372+0000) 2024-09-11T07:59:55.749 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 07:59:55 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T07:59:55.490+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:29.150096+0000 front 2024-09-11T07:59:29.149948+0000 (oldest deadline 2024-09-11T07:59:55.049814+0000) 2024-09-11T07:59:55.749 INFO:journalctl@ceph.osd.5.smithi139.stdout:Sep 11 07:59:55 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-5[37043]: 2024-09-11T07:59:55.505+0000 7f257a2e5640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:30.240233+0000 front 2024-09-11T07:59:30.240134+0000 (oldest deadline 2024-09-11T07:59:54.940194+0000) 2024-09-11T07:59:56.466 INFO:journalctl@ceph.osd.4.smithi139.stdout:Sep 11 07:59:56 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-4[33592]: 2024-09-11T07:59:56.011+0000 7fd5a86f5640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:30.198491+0000 front 2024-09-11T07:59:30.198479+0000 (oldest deadline 2024-09-11T07:59:54.298372+0000) 2024-09-11T07:59:56.748 INFO:journalctl@ceph.osd.5.smithi139.stdout:Sep 11 07:59:56 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-5[37043]: 2024-09-11T07:59:56.490+0000 7f257a2e5640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:30.240233+0000 front 2024-09-11T07:59:30.240134+0000 (oldest deadline 2024-09-11T07:59:54.940194+0000) 2024-09-11T07:59:56.749 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 07:59:56 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T07:59:56.464+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:29.150096+0000 front 2024-09-11T07:59:29.149948+0000 (oldest deadline 2024-09-11T07:59:55.049814+0000) 2024-09-11T07:59:57.364 INFO:journalctl@ceph.osd.4.smithi139.stdout:Sep 11 07:59:57 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-4[33592]: 2024-09-11T07:59:57.052+0000 7fd5a86f5640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:30.198491+0000 front 2024-09-11T07:59:30.198479+0000 (oldest deadline 2024-09-11T07:59:54.298372+0000) 2024-09-11T07:59:57.749 INFO:journalctl@ceph.osd.5.smithi139.stdout:Sep 11 07:59:57 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-5[37043]: 2024-09-11T07:59:57.502+0000 7f257a2e5640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:30.240233+0000 front 2024-09-11T07:59:30.240134+0000 (oldest deadline 2024-09-11T07:59:54.940194+0000) 2024-09-11T07:59:57.749 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 07:59:57 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T07:59:57.490+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:29.150096+0000 front 2024-09-11T07:59:29.149948+0000 (oldest deadline 2024-09-11T07:59:55.049814+0000) 2024-09-11T07:59:57.750 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 07:59:57 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6[40580]: 2024-09-11T07:59:57.362+0000 7f7cb1e5f640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:32.648956+0000 front 2024-09-11T07:59:32.648932+0000 (oldest deadline 2024-09-11T07:59:57.348856+0000) 2024-09-11T07:59:58.324 INFO:journalctl@ceph.osd.4.smithi139.stdout:Sep 11 07:59:58 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-4[33592]: 2024-09-11T07:59:58.095+0000 7fd5a86f5640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:30.198491+0000 front 2024-09-11T07:59:30.198479+0000 (oldest deadline 2024-09-11T07:59:54.298372+0000) 2024-09-11T07:59:58.686 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 07:59:58 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6[40580]: 2024-09-11T07:59:58.399+0000 7f7cb1e5f640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:32.648956+0000 front 2024-09-11T07:59:32.648932+0000 (oldest deadline 2024-09-11T07:59:57.348856+0000) 2024-09-11T07:59:58.687 INFO:journalctl@ceph.osd.5.smithi139.stdout:Sep 11 07:59:58 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-5[37043]: 2024-09-11T07:59:58.497+0000 7f257a2e5640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:30.240233+0000 front 2024-09-11T07:59:30.240134+0000 (oldest deadline 2024-09-11T07:59:54.940194+0000) 2024-09-11T07:59:58.687 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 07:59:58 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T07:59:58.462+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:29.150096+0000 front 2024-09-11T07:59:29.149948+0000 (oldest deadline 2024-09-11T07:59:55.049814+0000) 2024-09-11T07:59:58.718 INFO:journalctl@ceph.osd.3.smithi110.stdout:Sep 11 07:59:58 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-3[46734]: 2024-09-11T07:59:58.368+0000 7f1db9aa2640 -1 osd.3 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:32.309881+0000 front 2024-09-11T07:59:32.309871+0000 (oldest deadline 2024-09-11T07:59:58.209944+0000) 2024-09-11T07:59:59.343 INFO:journalctl@ceph.osd.4.smithi139.stdout:Sep 11 07:59:59 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-4[33592]: 2024-09-11T07:59:59.049+0000 7fd5a86f5640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:30.198491+0000 front 2024-09-11T07:59:30.198479+0000 (oldest deadline 2024-09-11T07:59:54.298372+0000) 2024-09-11T07:59:59.468 INFO:journalctl@ceph.osd.3.smithi110.stdout:Sep 11 07:59:59 smithi110 podman[63701]: 2024-09-11 07:59:59.214751454 +0000 UTC m=+5.504775541 container died b6bfac7bdfa630b2f7c91ca1306055c8183864150cdad3f53b7ffa8709234ae4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-11T07:59:59.748 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 07:59:59 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6[40580]: 2024-09-11T07:59:59.447+0000 7f7cb1e5f640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:32.648956+0000 front 2024-09-11T07:59:32.648932+0000 (oldest deadline 2024-09-11T07:59:57.348856+0000) 2024-09-11T07:59:59.749 INFO:journalctl@ceph.osd.5.smithi139.stdout:Sep 11 07:59:59 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-5[37043]: 2024-09-11T07:59:59.536+0000 7f257a2e5640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:30.240233+0000 front 2024-09-11T07:59:30.240134+0000 (oldest deadline 2024-09-11T07:59:54.940194+0000) 2024-09-11T07:59:59.749 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 07:59:59 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T07:59:59.416+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:29.150096+0000 front 2024-09-11T07:59:29.149948+0000 (oldest deadline 2024-09-11T07:59:55.049814+0000) 2024-09-11T07:59:59.931 INFO:journalctl@ceph.osd.3.smithi110.stdout:Sep 11 07:59:59 smithi110 podman[63701]: 2024-09-11 07:59:59.663139329 +0000 UTC m=+5.953163442 container cleanup b6bfac7bdfa630b2f7c91ca1306055c8183864150cdad3f53b7ffa8709234ae4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-3, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-11T07:59:59.931 INFO:journalctl@ceph.osd.3.smithi110.stdout:Sep 11 07:59:59 smithi110 bash[63701]: ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-3 2024-09-11T08:00:00.218 INFO:journalctl@ceph.osd.3.smithi110.stdout:Sep 11 07:59:59 smithi110 podman[63716]: 2024-09-11 07:59:59.930891439 +0000 UTC m=+0.711292348 container remove b6bfac7bdfa630b2f7c91ca1306055c8183864150cdad3f53b7ffa8709234ae4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-3, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0) 2024-09-11T08:00:00.398 INFO:journalctl@ceph.osd.4.smithi139.stdout:Sep 11 08:00:00 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-4[33592]: 2024-09-11T08:00:00.039+0000 7fd5a86f5640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:30.198491+0000 front 2024-09-11T07:59:30.198479+0000 (oldest deadline 2024-09-11T07:59:54.298372+0000) 2024-09-11T08:00:00.568 INFO:journalctl@ceph.osd.3.smithi110.stdout:Sep 11 08:00:00 smithi110 podman[63801]: 2024-09-11 08:00:00.265057132 +0000 UTC m=+0.106311319 container create 2803f7d613827147145d16a31a791a451e159370f6a49a1552387daa054816cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T08:00:00.568 INFO:journalctl@ceph.osd.3.smithi110.stdout:Sep 11 08:00:00 smithi110 podman[63801]: 2024-09-11 08:00:00.174635314 +0000 UTC m=+0.015889509 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T08:00:00.568 INFO:journalctl@ceph.osd.3.smithi110.stdout:Sep 11 08:00:00 smithi110 podman[63801]: 2024-09-11 08:00:00.501033702 +0000 UTC m=+0.342287889 container init 2803f7d613827147145d16a31a791a451e159370f6a49a1552387daa054816cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-11T08:00:00.569 INFO:journalctl@ceph.osd.3.smithi110.stdout:Sep 11 08:00:00 smithi110 podman[63801]: 2024-09-11 08:00:00.504502435 +0000 UTC m=+0.345756622 container start 2803f7d613827147145d16a31a791a451e159370f6a49a1552387daa054816cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-11T08:00:00.569 INFO:journalctl@ceph.osd.3.smithi110.stdout:Sep 11 08:00:00 smithi110 podman[63801]: 2024-09-11 08:00:00.567503211 +0000 UTC m=+0.408757405 container attach 2803f7d613827147145d16a31a791a451e159370f6a49a1552387daa054816cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-11T08:00:00.693 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 08:00:00 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6[40580]: 2024-09-11T08:00:00.494+0000 7f7cb1e5f640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:32.648956+0000 front 2024-09-11T07:59:32.648932+0000 (oldest deadline 2024-09-11T07:59:57.348856+0000) 2024-09-11T08:00:00.693 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:00 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:00.396+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:29.150096+0000 front 2024-09-11T07:59:29.149948+0000 (oldest deadline 2024-09-11T07:59:55.049814+0000) 2024-09-11T08:00:00.694 INFO:journalctl@ceph.osd.5.smithi139.stdout:Sep 11 08:00:00 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-5[37043]: 2024-09-11T08:00:00.559+0000 7f257a2e5640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:30.240233+0000 front 2024-09-11T07:59:30.240134+0000 (oldest deadline 2024-09-11T07:59:54.940194+0000) 2024-09-11T08:00:00.962 INFO:journalctl@ceph.osd.3.smithi110.stdout:Sep 11 08:00:00 smithi110 podman[63801]: 2024-09-11 08:00:00.682225355 +0000 UTC m=+0.523479535 container died 2803f7d613827147145d16a31a791a451e159370f6a49a1552387daa054816cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-11T08:00:01.248 INFO:journalctl@ceph.osd.4.smithi139.stdout:Sep 11 08:00:01 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-4[33592]: 2024-09-11T08:00:01.026+0000 7fd5a86f5640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:30.198491+0000 front 2024-09-11T07:59:30.198479+0000 (oldest deadline 2024-09-11T07:59:54.298372+0000) 2024-09-11T08:00:01.307 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-41bd1096-7012-11ef-bcea-c7b262605968@osd.3.service' 2024-09-11T08:00:01.347 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:00:01.347 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-11T08:00:01.347 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-11T08:00:01.347 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ceph-41bd1096-7012-11ef-bcea-c7b262605968@osd.4 2024-09-11T08:00:01.638 INFO:journalctl@ceph.osd.4.smithi139.stdout:Sep 11 08:00:01 smithi139 systemd[1]: Stopping Ceph osd.4 for 41bd1096-7012-11ef-bcea-c7b262605968... 2024-09-11T08:00:01.638 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 08:00:01 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6[40580]: 2024-09-11T08:00:01.482+0000 7f7cb1e5f640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:32.648956+0000 front 2024-09-11T07:59:32.648932+0000 (oldest deadline 2024-09-11T07:59:57.348856+0000) 2024-09-11T08:00:01.638 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:01 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:01.408+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:29.150096+0000 front 2024-09-11T07:59:29.149948+0000 (oldest deadline 2024-09-11T07:59:55.049814+0000) 2024-09-11T08:00:01.639 INFO:journalctl@ceph.osd.5.smithi139.stdout:Sep 11 08:00:01 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-5[37043]: 2024-09-11T08:00:01.597+0000 7f257a2e5640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:30.240233+0000 front 2024-09-11T07:59:30.240134+0000 (oldest deadline 2024-09-11T07:59:54.940194+0000) 2024-09-11T08:00:01.998 INFO:journalctl@ceph.osd.4.smithi139.stdout:Sep 11 08:00:01 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-4[33592]: 2024-09-11T08:00:01.636+0000 7fd5ab0db640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:00:01.999 INFO:journalctl@ceph.osd.4.smithi139.stdout:Sep 11 08:00:01 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-4[33592]: 2024-09-11T08:00:01.636+0000 7fd5ab0db640 -1 osd.4 71 *** Got signal Terminated *** 2024-09-11T08:00:01.999 INFO:journalctl@ceph.osd.4.smithi139.stdout:Sep 11 08:00:01 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-4[33592]: 2024-09-11T08:00:01.636+0000 7fd5ab0db640 -1 osd.4 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T08:00:02.410 INFO:journalctl@ceph.osd.4.smithi139.stdout:Sep 11 08:00:02 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-4[33592]: 2024-09-11T08:00:02.031+0000 7fd5a86f5640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:30.198491+0000 front 2024-09-11T07:59:30.198479+0000 (oldest deadline 2024-09-11T07:59:54.298372+0000) 2024-09-11T08:00:02.749 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:02 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:02.409+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:29.150096+0000 front 2024-09-11T07:59:29.149948+0000 (oldest deadline 2024-09-11T07:59:55.049814+0000) 2024-09-11T08:00:02.749 INFO:journalctl@ceph.osd.5.smithi139.stdout:Sep 11 08:00:02 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-5[37043]: 2024-09-11T08:00:02.574+0000 7f257a2e5640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:30.240233+0000 front 2024-09-11T07:59:30.240134+0000 (oldest deadline 2024-09-11T07:59:54.940194+0000) 2024-09-11T08:00:02.749 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 08:00:02 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6[40580]: 2024-09-11T08:00:02.451+0000 7f7cb1e5f640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:32.648956+0000 front 2024-09-11T07:59:32.648932+0000 (oldest deadline 2024-09-11T07:59:57.348856+0000) 2024-09-11T08:00:03.403 INFO:journalctl@ceph.osd.4.smithi139.stdout:Sep 11 08:00:03 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-4[33592]: 2024-09-11T08:00:03.010+0000 7fd5a86f5640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:30.198491+0000 front 2024-09-11T07:59:30.198479+0000 (oldest deadline 2024-09-11T07:59:54.298372+0000) 2024-09-11T08:00:03.403 INFO:journalctl@ceph.osd.4.smithi139.stdout:Sep 11 08:00:03 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-4[33592]: 2024-09-11T08:00:03.010+0000 7fd5a86f5640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-11T07:59:38.399337+0000 front 2024-09-11T07:59:38.399228+0000 (oldest deadline 2024-09-11T08:00:02.499163+0000) 2024-09-11T08:00:03.748 INFO:journalctl@ceph.osd.5.smithi139.stdout:Sep 11 08:00:03 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-5[37043]: 2024-09-11T08:00:03.617+0000 7f257a2e5640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:30.240233+0000 front 2024-09-11T07:59:30.240134+0000 (oldest deadline 2024-09-11T07:59:54.940194+0000) 2024-09-11T08:00:03.749 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 08:00:03 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6[40580]: 2024-09-11T08:00:03.470+0000 7f7cb1e5f640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:32.648956+0000 front 2024-09-11T07:59:32.648932+0000 (oldest deadline 2024-09-11T07:59:57.348856+0000) 2024-09-11T08:00:03.749 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 08:00:03 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6[40580]: 2024-09-11T08:00:03.470+0000 7f7cb1e5f640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-11T07:59:37.849843+0000 front 2024-09-11T07:59:37.849707+0000 (oldest deadline 2024-09-11T08:00:02.549536+0000) 2024-09-11T08:00:03.749 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:03 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:03.401+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:29.150096+0000 front 2024-09-11T07:59:29.149948+0000 (oldest deadline 2024-09-11T07:59:55.049814+0000) 2024-09-11T08:00:03.749 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:03 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:03.401+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-11T07:59:37.251440+0000 front 2024-09-11T07:59:37.251347+0000 (oldest deadline 2024-09-11T08:00:03.151225+0000) 2024-09-11T08:00:04.248 INFO:journalctl@ceph.osd.4.smithi139.stdout:Sep 11 08:00:03 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-4[33592]: 2024-09-11T08:00:03.977+0000 7fd5a86f5640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:30.198491+0000 front 2024-09-11T07:59:30.198479+0000 (oldest deadline 2024-09-11T07:59:54.298372+0000) 2024-09-11T08:00:04.249 INFO:journalctl@ceph.osd.4.smithi139.stdout:Sep 11 08:00:03 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-4[33592]: 2024-09-11T08:00:03.977+0000 7fd5a86f5640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-11T07:59:38.399337+0000 front 2024-09-11T07:59:38.399228+0000 (oldest deadline 2024-09-11T08:00:02.499163+0000) 2024-09-11T08:00:04.748 INFO:journalctl@ceph.osd.5.smithi139.stdout:Sep 11 08:00:04 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-5[37043]: 2024-09-11T08:00:04.620+0000 7f257a2e5640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:30.240233+0000 front 2024-09-11T07:59:30.240134+0000 (oldest deadline 2024-09-11T07:59:54.940194+0000) 2024-09-11T08:00:04.749 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 08:00:04 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6[40580]: 2024-09-11T08:00:04.430+0000 7f7cb1e5f640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:32.648956+0000 front 2024-09-11T07:59:32.648932+0000 (oldest deadline 2024-09-11T07:59:57.348856+0000) 2024-09-11T08:00:04.749 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 08:00:04 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6[40580]: 2024-09-11T08:00:04.430+0000 7f7cb1e5f640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-11T07:59:37.849843+0000 front 2024-09-11T07:59:37.849707+0000 (oldest deadline 2024-09-11T08:00:02.549536+0000) 2024-09-11T08:00:04.749 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:04 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:04.411+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:29.150096+0000 front 2024-09-11T07:59:29.149948+0000 (oldest deadline 2024-09-11T07:59:55.049814+0000) 2024-09-11T08:00:04.749 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:04 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:04.411+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-11T07:59:37.251440+0000 front 2024-09-11T07:59:37.251347+0000 (oldest deadline 2024-09-11T08:00:03.151225+0000) 2024-09-11T08:00:05.434 INFO:journalctl@ceph.osd.4.smithi139.stdout:Sep 11 08:00:05 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-4[33592]: 2024-09-11T08:00:05.005+0000 7fd5a86f5640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:30.198491+0000 front 2024-09-11T07:59:30.198479+0000 (oldest deadline 2024-09-11T07:59:54.298372+0000) 2024-09-11T08:00:05.434 INFO:journalctl@ceph.osd.4.smithi139.stdout:Sep 11 08:00:05 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-4[33592]: 2024-09-11T08:00:05.005+0000 7fd5a86f5640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-11T07:59:38.399337+0000 front 2024-09-11T07:59:38.399228+0000 (oldest deadline 2024-09-11T08:00:02.499163+0000) 2024-09-11T08:00:05.748 INFO:journalctl@ceph.osd.5.smithi139.stdout:Sep 11 08:00:05 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-5[37043]: 2024-09-11T08:00:05.593+0000 7f257a2e5640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:30.240233+0000 front 2024-09-11T07:59:30.240134+0000 (oldest deadline 2024-09-11T07:59:54.940194+0000) 2024-09-11T08:00:05.749 INFO:journalctl@ceph.osd.5.smithi139.stdout:Sep 11 08:00:05 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-5[37043]: 2024-09-11T08:00:05.593+0000 7f257a2e5640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-11T07:59:39.541718+0000 front 2024-09-11T07:59:39.541626+0000 (oldest deadline 2024-09-11T08:00:05.441559+0000) 2024-09-11T08:00:05.749 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 08:00:05 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6[40580]: 2024-09-11T08:00:05.447+0000 7f7cb1e5f640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:32.648956+0000 front 2024-09-11T07:59:32.648932+0000 (oldest deadline 2024-09-11T07:59:57.348856+0000) 2024-09-11T08:00:05.749 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 08:00:05 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6[40580]: 2024-09-11T08:00:05.447+0000 7f7cb1e5f640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-11T07:59:37.849843+0000 front 2024-09-11T07:59:37.849707+0000 (oldest deadline 2024-09-11T08:00:02.549536+0000) 2024-09-11T08:00:05.749 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:05 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:05.433+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:29.150096+0000 front 2024-09-11T07:59:29.149948+0000 (oldest deadline 2024-09-11T07:59:55.049814+0000) 2024-09-11T08:00:05.749 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:05 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:05.433+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-11T07:59:37.251440+0000 front 2024-09-11T07:59:37.251347+0000 (oldest deadline 2024-09-11T08:00:03.151225+0000) 2024-09-11T08:00:06.450 INFO:journalctl@ceph.osd.4.smithi139.stdout:Sep 11 08:00:06 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-4[33592]: 2024-09-11T08:00:06.039+0000 7fd5a86f5640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:30.198491+0000 front 2024-09-11T07:59:30.198479+0000 (oldest deadline 2024-09-11T07:59:54.298372+0000) 2024-09-11T08:00:06.451 INFO:journalctl@ceph.osd.4.smithi139.stdout:Sep 11 08:00:06 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-4[33592]: 2024-09-11T08:00:06.039+0000 7fd5a86f5640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-11T07:59:38.399337+0000 front 2024-09-11T07:59:38.399228+0000 (oldest deadline 2024-09-11T08:00:02.499163+0000) 2024-09-11T08:00:06.748 INFO:journalctl@ceph.osd.5.smithi139.stdout:Sep 11 08:00:06 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-5[37043]: 2024-09-11T08:00:06.625+0000 7f257a2e5640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:30.240233+0000 front 2024-09-11T07:59:30.240134+0000 (oldest deadline 2024-09-11T07:59:54.940194+0000) 2024-09-11T08:00:06.749 INFO:journalctl@ceph.osd.5.smithi139.stdout:Sep 11 08:00:06 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-5[37043]: 2024-09-11T08:00:06.625+0000 7f257a2e5640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-11T07:59:39.541718+0000 front 2024-09-11T07:59:39.541626+0000 (oldest deadline 2024-09-11T08:00:05.441559+0000) 2024-09-11T08:00:06.749 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 08:00:06 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6[40580]: 2024-09-11T08:00:06.468+0000 7f7cb1e5f640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:32.648956+0000 front 2024-09-11T07:59:32.648932+0000 (oldest deadline 2024-09-11T07:59:57.348856+0000) 2024-09-11T08:00:06.749 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 08:00:06 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6[40580]: 2024-09-11T08:00:06.468+0000 7f7cb1e5f640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-11T07:59:37.849843+0000 front 2024-09-11T07:59:37.849707+0000 (oldest deadline 2024-09-11T08:00:02.549536+0000) 2024-09-11T08:00:06.749 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:06 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:06.449+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:29.150096+0000 front 2024-09-11T07:59:29.149948+0000 (oldest deadline 2024-09-11T07:59:55.049814+0000) 2024-09-11T08:00:06.749 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:06 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:06.449+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-11T07:59:37.251440+0000 front 2024-09-11T07:59:37.251347+0000 (oldest deadline 2024-09-11T08:00:03.151225+0000) 2024-09-11T08:00:07.487 INFO:journalctl@ceph.osd.4.smithi139.stdout:Sep 11 08:00:07 smithi139 podman[55707]: 2024-09-11 08:00:07.185074383 +0000 UTC m=+5.651442589 container died 64fa5b25bebda91c048aa132e7441aec96904f494c0b8792bf5f54216b5933e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-4, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-11T08:00:07.746 INFO:journalctl@ceph.osd.4.smithi139.stdout:Sep 11 08:00:07 smithi139 podman[55707]: 2024-09-11 08:00:07.486808044 +0000 UTC m=+5.953176246 container cleanup 64fa5b25bebda91c048aa132e7441aec96904f494c0b8792bf5f54216b5933e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-11T08:00:07.747 INFO:journalctl@ceph.osd.4.smithi139.stdout:Sep 11 08:00:07 smithi139 bash[55707]: ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-4 2024-09-11T08:00:07.747 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 08:00:07 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6[40580]: 2024-09-11T08:00:07.491+0000 7f7cb1e5f640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:32.648956+0000 front 2024-09-11T07:59:32.648932+0000 (oldest deadline 2024-09-11T07:59:57.348856+0000) 2024-09-11T08:00:07.747 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 08:00:07 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6[40580]: 2024-09-11T08:00:07.491+0000 7f7cb1e5f640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-11T07:59:37.849843+0000 front 2024-09-11T07:59:37.849707+0000 (oldest deadline 2024-09-11T08:00:02.549536+0000) 2024-09-11T08:00:07.747 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:07 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:07.487+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:29.150096+0000 front 2024-09-11T07:59:29.149948+0000 (oldest deadline 2024-09-11T07:59:55.049814+0000) 2024-09-11T08:00:07.748 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:07 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:07.487+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-11T07:59:37.251440+0000 front 2024-09-11T07:59:37.251347+0000 (oldest deadline 2024-09-11T08:00:03.151225+0000) 2024-09-11T08:00:07.748 INFO:journalctl@ceph.osd.5.smithi139.stdout:Sep 11 08:00:07 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-5[37043]: 2024-09-11T08:00:07.586+0000 7f257a2e5640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:30.240233+0000 front 2024-09-11T07:59:30.240134+0000 (oldest deadline 2024-09-11T07:59:54.940194+0000) 2024-09-11T08:00:07.748 INFO:journalctl@ceph.osd.5.smithi139.stdout:Sep 11 08:00:07 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-5[37043]: 2024-09-11T08:00:07.586+0000 7f257a2e5640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-11T07:59:39.541718+0000 front 2024-09-11T07:59:39.541626+0000 (oldest deadline 2024-09-11T08:00:05.441559+0000) 2024-09-11T08:00:07.998 INFO:journalctl@ceph.osd.4.smithi139.stdout:Sep 11 08:00:07 smithi139 podman[55724]: 2024-09-11 08:00:07.74633467 +0000 UTC m=+0.555031546 container remove 64fa5b25bebda91c048aa132e7441aec96904f494c0b8792bf5f54216b5933e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T08:00:08.400 INFO:journalctl@ceph.osd.4.smithi139.stdout:Sep 11 08:00:08 smithi139 podman[55817]: 2024-09-11 08:00:08.000536629 +0000 UTC m=+0.015731009 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T08:00:08.400 INFO:journalctl@ceph.osd.4.smithi139.stdout:Sep 11 08:00:08 smithi139 podman[55817]: 2024-09-11 08:00:08.114158581 +0000 UTC m=+0.129352993 container create 4a5c465b224e7c975e21d00fe465224bbb0a13ef092b8e2f2f2c42a84803da54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS) 2024-09-11T08:00:08.400 INFO:journalctl@ceph.osd.4.smithi139.stdout:Sep 11 08:00:08 smithi139 podman[55817]: 2024-09-11 08:00:08.332993907 +0000 UTC m=+0.348188277 container init 4a5c465b224e7c975e21d00fe465224bbb0a13ef092b8e2f2f2c42a84803da54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-4-deactivate, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731) 2024-09-11T08:00:08.400 INFO:journalctl@ceph.osd.4.smithi139.stdout:Sep 11 08:00:08 smithi139 podman[55817]: 2024-09-11 08:00:08.336438573 +0000 UTC m=+0.351632940 container start 4a5c465b224e7c975e21d00fe465224bbb0a13ef092b8e2f2f2c42a84803da54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-11T08:00:08.400 INFO:journalctl@ceph.osd.4.smithi139.stdout:Sep 11 08:00:08 smithi139 podman[55817]: 2024-09-11 08:00:08.39955045 +0000 UTC m=+0.414744832 container attach 4a5c465b224e7c975e21d00fe465224bbb0a13ef092b8e2f2f2c42a84803da54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T08:00:08.749 INFO:journalctl@ceph.osd.4.smithi139.stdout:Sep 11 08:00:08 smithi139 podman[55817]: 2024-09-11 08:00:08.519200984 +0000 UTC m=+0.534395362 container died 4a5c465b224e7c975e21d00fe465224bbb0a13ef092b8e2f2f2c42a84803da54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-11T08:00:08.749 INFO:journalctl@ceph.osd.5.smithi139.stdout:Sep 11 08:00:08 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-5[37043]: 2024-09-11T08:00:08.621+0000 7f257a2e5640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:30.240233+0000 front 2024-09-11T07:59:30.240134+0000 (oldest deadline 2024-09-11T07:59:54.940194+0000) 2024-09-11T08:00:08.749 INFO:journalctl@ceph.osd.5.smithi139.stdout:Sep 11 08:00:08 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-5[37043]: 2024-09-11T08:00:08.621+0000 7f257a2e5640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-11T07:59:39.541718+0000 front 2024-09-11T07:59:39.541626+0000 (oldest deadline 2024-09-11T08:00:05.441559+0000) 2024-09-11T08:00:08.749 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 08:00:08 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6[40580]: 2024-09-11T08:00:08.495+0000 7f7cb1e5f640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:32.648956+0000 front 2024-09-11T07:59:32.648932+0000 (oldest deadline 2024-09-11T07:59:57.348856+0000) 2024-09-11T08:00:08.750 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 08:00:08 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6[40580]: 2024-09-11T08:00:08.495+0000 7f7cb1e5f640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-11T07:59:37.849843+0000 front 2024-09-11T07:59:37.849707+0000 (oldest deadline 2024-09-11T08:00:02.549536+0000) 2024-09-11T08:00:08.750 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:08 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:08.505+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:29.150096+0000 front 2024-09-11T07:59:29.149948+0000 (oldest deadline 2024-09-11T07:59:55.049814+0000) 2024-09-11T08:00:08.750 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:08 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:08.505+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-11T07:59:37.251440+0000 front 2024-09-11T07:59:37.251347+0000 (oldest deadline 2024-09-11T08:00:03.151225+0000) 2024-09-11T08:00:09.144 DEBUG:teuthology.orchestra.run.smithi139:> sudo pkill -f 'journalctl -f -n 0 -u ceph-41bd1096-7012-11ef-bcea-c7b262605968@osd.4.service' 2024-09-11T08:00:09.184 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:00:09.184 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-11T08:00:09.184 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-11T08:00:09.184 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ceph-41bd1096-7012-11ef-bcea-c7b262605968@osd.5 2024-09-11T08:00:09.498 INFO:journalctl@ceph.osd.5.smithi139.stdout:Sep 11 08:00:09 smithi139 systemd[1]: Stopping Ceph osd.5 for 41bd1096-7012-11ef-bcea-c7b262605968... 2024-09-11T08:00:09.998 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:09 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:09.501+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:29.150096+0000 front 2024-09-11T07:59:29.149948+0000 (oldest deadline 2024-09-11T07:59:55.049814+0000) 2024-09-11T08:00:09.999 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:09 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:09.501+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-11T07:59:37.251440+0000 front 2024-09-11T07:59:37.251347+0000 (oldest deadline 2024-09-11T08:00:03.151225+0000) 2024-09-11T08:00:09.999 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 08:00:09 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6[40580]: 2024-09-11T08:00:09.543+0000 7f7cb1e5f640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:32.648956+0000 front 2024-09-11T07:59:32.648932+0000 (oldest deadline 2024-09-11T07:59:57.348856+0000) 2024-09-11T08:00:09.999 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 08:00:09 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6[40580]: 2024-09-11T08:00:09.543+0000 7f7cb1e5f640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-11T07:59:37.849843+0000 front 2024-09-11T07:59:37.849707+0000 (oldest deadline 2024-09-11T08:00:02.549536+0000) 2024-09-11T08:00:09.999 INFO:journalctl@ceph.osd.5.smithi139.stdout:Sep 11 08:00:09 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-5[37043]: 2024-09-11T08:00:09.497+0000 7f257eccf640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:00:09.999 INFO:journalctl@ceph.osd.5.smithi139.stdout:Sep 11 08:00:09 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-5[37043]: 2024-09-11T08:00:09.497+0000 7f257eccf640 -1 osd.5 71 *** Got signal Terminated *** 2024-09-11T08:00:09.999 INFO:journalctl@ceph.osd.5.smithi139.stdout:Sep 11 08:00:09 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-5[37043]: 2024-09-11T08:00:09.497+0000 7f257eccf640 -1 osd.5 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T08:00:09.999 INFO:journalctl@ceph.osd.5.smithi139.stdout:Sep 11 08:00:09 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-5[37043]: 2024-09-11T08:00:09.584+0000 7f257a2e5640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:30.240233+0000 front 2024-09-11T07:59:30.240134+0000 (oldest deadline 2024-09-11T07:59:54.940194+0000) 2024-09-11T08:00:10.000 INFO:journalctl@ceph.osd.5.smithi139.stdout:Sep 11 08:00:09 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-5[37043]: 2024-09-11T08:00:09.584+0000 7f257a2e5640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-11T07:59:39.541718+0000 front 2024-09-11T07:59:39.541626+0000 (oldest deadline 2024-09-11T08:00:05.441559+0000) 2024-09-11T08:00:10.998 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:10 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:10.529+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:29.150096+0000 front 2024-09-11T07:59:29.149948+0000 (oldest deadline 2024-09-11T07:59:55.049814+0000) 2024-09-11T08:00:10.999 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:10 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:10.529+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-11T07:59:37.251440+0000 front 2024-09-11T07:59:37.251347+0000 (oldest deadline 2024-09-11T08:00:03.151225+0000) 2024-09-11T08:00:10.999 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 08:00:10 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6[40580]: 2024-09-11T08:00:10.577+0000 7f7cb1e5f640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:32.648956+0000 front 2024-09-11T07:59:32.648932+0000 (oldest deadline 2024-09-11T07:59:57.348856+0000) 2024-09-11T08:00:10.999 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 08:00:10 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6[40580]: 2024-09-11T08:00:10.577+0000 7f7cb1e5f640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-11T07:59:37.849843+0000 front 2024-09-11T07:59:37.849707+0000 (oldest deadline 2024-09-11T08:00:02.549536+0000) 2024-09-11T08:00:10.999 INFO:journalctl@ceph.osd.5.smithi139.stdout:Sep 11 08:00:10 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-5[37043]: 2024-09-11T08:00:10.541+0000 7f257a2e5640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:30.240233+0000 front 2024-09-11T07:59:30.240134+0000 (oldest deadline 2024-09-11T07:59:54.940194+0000) 2024-09-11T08:00:10.999 INFO:journalctl@ceph.osd.5.smithi139.stdout:Sep 11 08:00:10 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-5[37043]: 2024-09-11T08:00:10.541+0000 7f257a2e5640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-11T07:59:39.541718+0000 front 2024-09-11T07:59:39.541626+0000 (oldest deadline 2024-09-11T08:00:05.441559+0000) 2024-09-11T08:00:11.749 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 08:00:11 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6[40580]: 2024-09-11T08:00:11.547+0000 7f7cb1e5f640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:32.648956+0000 front 2024-09-11T07:59:32.648932+0000 (oldest deadline 2024-09-11T07:59:57.348856+0000) 2024-09-11T08:00:11.749 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 08:00:11 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6[40580]: 2024-09-11T08:00:11.547+0000 7f7cb1e5f640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-11T07:59:37.849843+0000 front 2024-09-11T07:59:37.849707+0000 (oldest deadline 2024-09-11T08:00:02.549536+0000) 2024-09-11T08:00:11.749 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 08:00:11 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6[40580]: 2024-09-11T08:00:11.547+0000 7f7cb1e5f640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.110:6822 osd.2 since back 2024-09-11T07:59:47.051305+0000 front 2024-09-11T07:59:47.051207+0000 (oldest deadline 2024-09-11T08:00:11.151093+0000) 2024-09-11T08:00:11.749 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:11 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:11.495+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:29.150096+0000 front 2024-09-11T07:59:29.149948+0000 (oldest deadline 2024-09-11T07:59:55.049814+0000) 2024-09-11T08:00:11.749 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:11 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:11.495+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-11T07:59:37.251440+0000 front 2024-09-11T07:59:37.251347+0000 (oldest deadline 2024-09-11T08:00:03.151225+0000) 2024-09-11T08:00:11.749 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:11 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:11.495+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6822 osd.2 since back 2024-09-11T07:59:46.052137+0000 front 2024-09-11T07:59:46.052023+0000 (oldest deadline 2024-09-11T08:00:10.751990+0000) 2024-09-11T08:00:11.750 INFO:journalctl@ceph.osd.5.smithi139.stdout:Sep 11 08:00:11 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-5[37043]: 2024-09-11T08:00:11.588+0000 7f257a2e5640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:30.240233+0000 front 2024-09-11T07:59:30.240134+0000 (oldest deadline 2024-09-11T07:59:54.940194+0000) 2024-09-11T08:00:11.750 INFO:journalctl@ceph.osd.5.smithi139.stdout:Sep 11 08:00:11 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-5[37043]: 2024-09-11T08:00:11.588+0000 7f257a2e5640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-11T07:59:39.541718+0000 front 2024-09-11T07:59:39.541626+0000 (oldest deadline 2024-09-11T08:00:05.441559+0000) 2024-09-11T08:00:12.749 INFO:journalctl@ceph.osd.5.smithi139.stdout:Sep 11 08:00:12 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-5[37043]: 2024-09-11T08:00:12.637+0000 7f257a2e5640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:30.240233+0000 front 2024-09-11T07:59:30.240134+0000 (oldest deadline 2024-09-11T07:59:54.940194+0000) 2024-09-11T08:00:12.749 INFO:journalctl@ceph.osd.5.smithi139.stdout:Sep 11 08:00:12 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-5[37043]: 2024-09-11T08:00:12.637+0000 7f257a2e5640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-11T07:59:39.541718+0000 front 2024-09-11T07:59:39.541626+0000 (oldest deadline 2024-09-11T08:00:05.441559+0000) 2024-09-11T08:00:12.749 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 08:00:12 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6[40580]: 2024-09-11T08:00:12.584+0000 7f7cb1e5f640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:32.648956+0000 front 2024-09-11T07:59:32.648932+0000 (oldest deadline 2024-09-11T07:59:57.348856+0000) 2024-09-11T08:00:12.749 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 08:00:12 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6[40580]: 2024-09-11T08:00:12.584+0000 7f7cb1e5f640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-11T07:59:37.849843+0000 front 2024-09-11T07:59:37.849707+0000 (oldest deadline 2024-09-11T08:00:02.549536+0000) 2024-09-11T08:00:12.749 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 08:00:12 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6[40580]: 2024-09-11T08:00:12.584+0000 7f7cb1e5f640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.110:6822 osd.2 since back 2024-09-11T07:59:47.051305+0000 front 2024-09-11T07:59:47.051207+0000 (oldest deadline 2024-09-11T08:00:11.151093+0000) 2024-09-11T08:00:12.749 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:12 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:12.467+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:29.150096+0000 front 2024-09-11T07:59:29.149948+0000 (oldest deadline 2024-09-11T07:59:55.049814+0000) 2024-09-11T08:00:12.750 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:12 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:12.467+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-11T07:59:37.251440+0000 front 2024-09-11T07:59:37.251347+0000 (oldest deadline 2024-09-11T08:00:03.151225+0000) 2024-09-11T08:00:12.750 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:12 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:12.467+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6822 osd.2 since back 2024-09-11T07:59:46.052137+0000 front 2024-09-11T07:59:46.052023+0000 (oldest deadline 2024-09-11T08:00:10.751990+0000) 2024-09-11T08:00:13.749 INFO:journalctl@ceph.osd.5.smithi139.stdout:Sep 11 08:00:13 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-5[37043]: 2024-09-11T08:00:13.604+0000 7f257a2e5640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:30.240233+0000 front 2024-09-11T07:59:30.240134+0000 (oldest deadline 2024-09-11T07:59:54.940194+0000) 2024-09-11T08:00:13.749 INFO:journalctl@ceph.osd.5.smithi139.stdout:Sep 11 08:00:13 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-5[37043]: 2024-09-11T08:00:13.604+0000 7f257a2e5640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-11T07:59:39.541718+0000 front 2024-09-11T07:59:39.541626+0000 (oldest deadline 2024-09-11T08:00:05.441559+0000) 2024-09-11T08:00:13.749 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 08:00:13 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6[40580]: 2024-09-11T08:00:13.604+0000 7f7cb1e5f640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:32.648956+0000 front 2024-09-11T07:59:32.648932+0000 (oldest deadline 2024-09-11T07:59:57.348856+0000) 2024-09-11T08:00:13.749 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 08:00:13 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6[40580]: 2024-09-11T08:00:13.604+0000 7f7cb1e5f640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-11T07:59:37.849843+0000 front 2024-09-11T07:59:37.849707+0000 (oldest deadline 2024-09-11T08:00:02.549536+0000) 2024-09-11T08:00:13.749 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 08:00:13 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6[40580]: 2024-09-11T08:00:13.604+0000 7f7cb1e5f640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.110:6822 osd.2 since back 2024-09-11T07:59:47.051305+0000 front 2024-09-11T07:59:47.051207+0000 (oldest deadline 2024-09-11T08:00:11.151093+0000) 2024-09-11T08:00:13.750 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:13 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:13.485+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:29.150096+0000 front 2024-09-11T07:59:29.149948+0000 (oldest deadline 2024-09-11T07:59:55.049814+0000) 2024-09-11T08:00:13.750 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:13 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:13.485+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-11T07:59:37.251440+0000 front 2024-09-11T07:59:37.251347+0000 (oldest deadline 2024-09-11T08:00:03.151225+0000) 2024-09-11T08:00:13.750 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:13 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:13.485+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6822 osd.2 since back 2024-09-11T07:59:46.052137+0000 front 2024-09-11T07:59:46.052023+0000 (oldest deadline 2024-09-11T08:00:10.751990+0000) 2024-09-11T08:00:14.817 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 08:00:14 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6[40580]: 2024-09-11T08:00:14.601+0000 7f7cb1e5f640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:32.648956+0000 front 2024-09-11T07:59:32.648932+0000 (oldest deadline 2024-09-11T07:59:57.348856+0000) 2024-09-11T08:00:14.817 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 08:00:14 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6[40580]: 2024-09-11T08:00:14.601+0000 7f7cb1e5f640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-11T07:59:37.849843+0000 front 2024-09-11T07:59:37.849707+0000 (oldest deadline 2024-09-11T08:00:02.549536+0000) 2024-09-11T08:00:14.817 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 08:00:14 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6[40580]: 2024-09-11T08:00:14.601+0000 7f7cb1e5f640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.110:6822 osd.2 since back 2024-09-11T07:59:47.051305+0000 front 2024-09-11T07:59:47.051207+0000 (oldest deadline 2024-09-11T08:00:11.151093+0000) 2024-09-11T08:00:14.817 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:14 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:14.501+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:29.150096+0000 front 2024-09-11T07:59:29.149948+0000 (oldest deadline 2024-09-11T07:59:55.049814+0000) 2024-09-11T08:00:14.818 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:14 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:14.501+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-11T07:59:37.251440+0000 front 2024-09-11T07:59:37.251347+0000 (oldest deadline 2024-09-11T08:00:03.151225+0000) 2024-09-11T08:00:14.818 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:14 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:14.501+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6822 osd.2 since back 2024-09-11T07:59:46.052137+0000 front 2024-09-11T07:59:46.052023+0000 (oldest deadline 2024-09-11T08:00:10.751990+0000) 2024-09-11T08:00:15.240 INFO:journalctl@ceph.osd.5.smithi139.stdout:Sep 11 08:00:14 smithi139 podman[55920]: 2024-09-11 08:00:14.951106022 +0000 UTC m=+5.545149311 container died 70d5d2bdbd5bf5837fb2bdf9865196685d95beac7af15739a664a008357ebdad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-5, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-11T08:00:15.498 INFO:journalctl@ceph.osd.5.smithi139.stdout:Sep 11 08:00:15 smithi139 podman[55920]: 2024-09-11 08:00:15.405489828 +0000 UTC m=+5.999533117 container cleanup 70d5d2bdbd5bf5837fb2bdf9865196685d95beac7af15739a664a008357ebdad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-5, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=) 2024-09-11T08:00:15.499 INFO:journalctl@ceph.osd.5.smithi139.stdout:Sep 11 08:00:15 smithi139 bash[55920]: ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-5 2024-09-11T08:00:15.499 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:15 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:15.457+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:29.150096+0000 front 2024-09-11T07:59:29.149948+0000 (oldest deadline 2024-09-11T07:59:55.049814+0000) 2024-09-11T08:00:15.499 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:15 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:15.457+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-11T07:59:37.251440+0000 front 2024-09-11T07:59:37.251347+0000 (oldest deadline 2024-09-11T08:00:03.151225+0000) 2024-09-11T08:00:15.499 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:15 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:15.457+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6822 osd.2 since back 2024-09-11T07:59:46.052137+0000 front 2024-09-11T07:59:46.052023+0000 (oldest deadline 2024-09-11T08:00:10.751990+0000) 2024-09-11T08:00:15.957 INFO:journalctl@ceph.osd.5.smithi139.stdout:Sep 11 08:00:15 smithi139 podman[55948]: 2024-09-11 08:00:15.640224034 +0000 UTC m=+0.683088815 container remove 70d5d2bdbd5bf5837fb2bdf9865196685d95beac7af15739a664a008357ebdad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-5, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T08:00:15.957 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 08:00:15 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6[40580]: 2024-09-11T08:00:15.614+0000 7f7cb1e5f640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:32.648956+0000 front 2024-09-11T07:59:32.648932+0000 (oldest deadline 2024-09-11T07:59:57.348856+0000) 2024-09-11T08:00:15.957 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 08:00:15 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6[40580]: 2024-09-11T08:00:15.614+0000 7f7cb1e5f640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-11T07:59:37.849843+0000 front 2024-09-11T07:59:37.849707+0000 (oldest deadline 2024-09-11T08:00:02.549536+0000) 2024-09-11T08:00:15.958 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 08:00:15 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6[40580]: 2024-09-11T08:00:15.614+0000 7f7cb1e5f640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.110:6822 osd.2 since back 2024-09-11T07:59:47.051305+0000 front 2024-09-11T07:59:47.051207+0000 (oldest deadline 2024-09-11T08:00:11.151093+0000) 2024-09-11T08:00:16.211 INFO:journalctl@ceph.osd.5.smithi139.stdout:Sep 11 08:00:15 smithi139 podman[56037]: 2024-09-11 08:00:15.859551602 +0000 UTC m=+0.015529639 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T08:00:16.211 INFO:journalctl@ceph.osd.5.smithi139.stdout:Sep 11 08:00:15 smithi139 podman[56037]: 2024-09-11 08:00:15.980390067 +0000 UTC m=+0.136368150 container create 711ac24a8802f58c6421510fdc91fa141d88fe7bd866a6a3502bf976aa130865 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T08:00:16.499 INFO:journalctl@ceph.osd.5.smithi139.stdout:Sep 11 08:00:16 smithi139 podman[56037]: 2024-09-11 08:00:16.210641268 +0000 UTC m=+0.366619305 container init 711ac24a8802f58c6421510fdc91fa141d88fe7bd866a6a3502bf976aa130865 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-11T08:00:16.499 INFO:journalctl@ceph.osd.5.smithi139.stdout:Sep 11 08:00:16 smithi139 podman[56037]: 2024-09-11 08:00:16.214163294 +0000 UTC m=+0.370141332 container start 711ac24a8802f58c6421510fdc91fa141d88fe7bd866a6a3502bf976aa130865 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-11T08:00:16.499 INFO:journalctl@ceph.osd.5.smithi139.stdout:Sep 11 08:00:16 smithi139 podman[56037]: 2024-09-11 08:00:16.293752375 +0000 UTC m=+0.449730417 container attach 711ac24a8802f58c6421510fdc91fa141d88fe7bd866a6a3502bf976aa130865 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-11T08:00:16.499 INFO:journalctl@ceph.osd.5.smithi139.stdout:Sep 11 08:00:16 smithi139 podman[56037]: 2024-09-11 08:00:16.394286271 +0000 UTC m=+0.550264317 container died 711ac24a8802f58c6421510fdc91fa141d88fe7bd866a6a3502bf976aa130865 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T08:00:16.500 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:16 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:16.458+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:29.150096+0000 front 2024-09-11T07:59:29.149948+0000 (oldest deadline 2024-09-11T07:59:55.049814+0000) 2024-09-11T08:00:16.500 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:16 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:16.458+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-11T07:59:37.251440+0000 front 2024-09-11T07:59:37.251347+0000 (oldest deadline 2024-09-11T08:00:03.151225+0000) 2024-09-11T08:00:16.500 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:16 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:16.458+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6822 osd.2 since back 2024-09-11T07:59:46.052137+0000 front 2024-09-11T07:59:46.052023+0000 (oldest deadline 2024-09-11T08:00:10.751990+0000) 2024-09-11T08:00:16.999 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 08:00:16 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6[40580]: 2024-09-11T08:00:16.601+0000 7f7cb1e5f640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:32.648956+0000 front 2024-09-11T07:59:32.648932+0000 (oldest deadline 2024-09-11T07:59:57.348856+0000) 2024-09-11T08:00:16.999 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 08:00:16 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6[40580]: 2024-09-11T08:00:16.601+0000 7f7cb1e5f640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-11T07:59:37.849843+0000 front 2024-09-11T07:59:37.849707+0000 (oldest deadline 2024-09-11T08:00:02.549536+0000) 2024-09-11T08:00:16.999 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 08:00:16 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6[40580]: 2024-09-11T08:00:16.601+0000 7f7cb1e5f640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.110:6822 osd.2 since back 2024-09-11T07:59:47.051305+0000 front 2024-09-11T07:59:47.051207+0000 (oldest deadline 2024-09-11T08:00:11.151093+0000) 2024-09-11T08:00:17.032 DEBUG:teuthology.orchestra.run.smithi139:> sudo pkill -f 'journalctl -f -n 0 -u ceph-41bd1096-7012-11ef-bcea-c7b262605968@osd.5.service' 2024-09-11T08:00:17.072 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:00:17.072 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-11T08:00:17.073 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-11T08:00:17.073 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ceph-41bd1096-7012-11ef-bcea-c7b262605968@osd.6 2024-09-11T08:00:17.401 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 08:00:17 smithi139 systemd[1]: Stopping Ceph osd.6 for 41bd1096-7012-11ef-bcea-c7b262605968... 2024-09-11T08:00:17.749 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:17 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:17.500+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:29.150096+0000 front 2024-09-11T07:59:29.149948+0000 (oldest deadline 2024-09-11T07:59:55.049814+0000) 2024-09-11T08:00:17.749 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:17 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:17.500+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-11T07:59:37.251440+0000 front 2024-09-11T07:59:37.251347+0000 (oldest deadline 2024-09-11T08:00:03.151225+0000) 2024-09-11T08:00:17.749 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:17 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:17.500+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6822 osd.2 since back 2024-09-11T07:59:46.052137+0000 front 2024-09-11T07:59:46.052023+0000 (oldest deadline 2024-09-11T08:00:10.751990+0000) 2024-09-11T08:00:17.749 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 08:00:17 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6[40580]: 2024-09-11T08:00:17.399+0000 7f7cb884d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:00:17.749 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 08:00:17 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6[40580]: 2024-09-11T08:00:17.399+0000 7f7cb884d640 -1 osd.6 71 *** Got signal Terminated *** 2024-09-11T08:00:17.749 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 08:00:17 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6[40580]: 2024-09-11T08:00:17.399+0000 7f7cb884d640 -1 osd.6 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T08:00:17.749 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 08:00:17 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6[40580]: 2024-09-11T08:00:17.607+0000 7f7cb1e5f640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:32.648956+0000 front 2024-09-11T07:59:32.648932+0000 (oldest deadline 2024-09-11T07:59:57.348856+0000) 2024-09-11T08:00:17.749 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 08:00:17 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6[40580]: 2024-09-11T08:00:17.607+0000 7f7cb1e5f640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-11T07:59:37.849843+0000 front 2024-09-11T07:59:37.849707+0000 (oldest deadline 2024-09-11T08:00:02.549536+0000) 2024-09-11T08:00:17.749 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 08:00:17 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6[40580]: 2024-09-11T08:00:17.607+0000 7f7cb1e5f640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.110:6822 osd.2 since back 2024-09-11T07:59:47.051305+0000 front 2024-09-11T07:59:47.051207+0000 (oldest deadline 2024-09-11T08:00:11.151093+0000) 2024-09-11T08:00:18.749 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:18 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:18.471+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:29.150096+0000 front 2024-09-11T07:59:29.149948+0000 (oldest deadline 2024-09-11T07:59:55.049814+0000) 2024-09-11T08:00:18.749 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:18 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:18.471+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-11T07:59:37.251440+0000 front 2024-09-11T07:59:37.251347+0000 (oldest deadline 2024-09-11T08:00:03.151225+0000) 2024-09-11T08:00:18.749 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:18 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:18.471+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6822 osd.2 since back 2024-09-11T07:59:46.052137+0000 front 2024-09-11T07:59:46.052023+0000 (oldest deadline 2024-09-11T08:00:10.751990+0000) 2024-09-11T08:00:18.749 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 08:00:18 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6[40580]: 2024-09-11T08:00:18.561+0000 7f7cb1e5f640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:32.648956+0000 front 2024-09-11T07:59:32.648932+0000 (oldest deadline 2024-09-11T07:59:57.348856+0000) 2024-09-11T08:00:18.749 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 08:00:18 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6[40580]: 2024-09-11T08:00:18.561+0000 7f7cb1e5f640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-11T07:59:37.849843+0000 front 2024-09-11T07:59:37.849707+0000 (oldest deadline 2024-09-11T08:00:02.549536+0000) 2024-09-11T08:00:18.749 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 08:00:18 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6[40580]: 2024-09-11T08:00:18.561+0000 7f7cb1e5f640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.110:6822 osd.2 since back 2024-09-11T07:59:47.051305+0000 front 2024-09-11T07:59:47.051207+0000 (oldest deadline 2024-09-11T08:00:11.151093+0000) 2024-09-11T08:00:19.749 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 08:00:19 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6[40580]: 2024-09-11T08:00:19.543+0000 7f7cb1e5f640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:32.648956+0000 front 2024-09-11T07:59:32.648932+0000 (oldest deadline 2024-09-11T07:59:57.348856+0000) 2024-09-11T08:00:19.749 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 08:00:19 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6[40580]: 2024-09-11T08:00:19.543+0000 7f7cb1e5f640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-11T07:59:37.849843+0000 front 2024-09-11T07:59:37.849707+0000 (oldest deadline 2024-09-11T08:00:02.549536+0000) 2024-09-11T08:00:19.749 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 08:00:19 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6[40580]: 2024-09-11T08:00:19.543+0000 7f7cb1e5f640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.110:6822 osd.2 since back 2024-09-11T07:59:47.051305+0000 front 2024-09-11T07:59:47.051207+0000 (oldest deadline 2024-09-11T08:00:11.151093+0000) 2024-09-11T08:00:19.750 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:19 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:19.484+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:29.150096+0000 front 2024-09-11T07:59:29.149948+0000 (oldest deadline 2024-09-11T07:59:55.049814+0000) 2024-09-11T08:00:19.750 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:19 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:19.484+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-11T07:59:37.251440+0000 front 2024-09-11T07:59:37.251347+0000 (oldest deadline 2024-09-11T08:00:03.151225+0000) 2024-09-11T08:00:19.750 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:19 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:19.484+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6822 osd.2 since back 2024-09-11T07:59:46.052137+0000 front 2024-09-11T07:59:46.052023+0000 (oldest deadline 2024-09-11T08:00:10.751990+0000) 2024-09-11T08:00:20.528 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:20 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:20.477+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:29.150096+0000 front 2024-09-11T07:59:29.149948+0000 (oldest deadline 2024-09-11T07:59:55.049814+0000) 2024-09-11T08:00:20.528 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:20 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:20.477+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-11T07:59:37.251440+0000 front 2024-09-11T07:59:37.251347+0000 (oldest deadline 2024-09-11T08:00:03.151225+0000) 2024-09-11T08:00:20.528 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:20 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:20.477+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6822 osd.2 since back 2024-09-11T07:59:46.052137+0000 front 2024-09-11T07:59:46.052023+0000 (oldest deadline 2024-09-11T08:00:10.751990+0000) 2024-09-11T08:00:20.998 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 08:00:20 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6[40580]: 2024-09-11T08:00:20.526+0000 7f7cb1e5f640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:32.648956+0000 front 2024-09-11T07:59:32.648932+0000 (oldest deadline 2024-09-11T07:59:57.348856+0000) 2024-09-11T08:00:20.998 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 08:00:20 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6[40580]: 2024-09-11T08:00:20.526+0000 7f7cb1e5f640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-11T07:59:37.849843+0000 front 2024-09-11T07:59:37.849707+0000 (oldest deadline 2024-09-11T08:00:02.549536+0000) 2024-09-11T08:00:20.999 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 08:00:20 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6[40580]: 2024-09-11T08:00:20.526+0000 7f7cb1e5f640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.110:6822 osd.2 since back 2024-09-11T07:59:47.051305+0000 front 2024-09-11T07:59:47.051207+0000 (oldest deadline 2024-09-11T08:00:11.151093+0000) 2024-09-11T08:00:21.749 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 08:00:21 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6[40580]: 2024-09-11T08:00:21.528+0000 7f7cb1e5f640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:32.648956+0000 front 2024-09-11T07:59:32.648932+0000 (oldest deadline 2024-09-11T07:59:57.348856+0000) 2024-09-11T08:00:21.749 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 08:00:21 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6[40580]: 2024-09-11T08:00:21.528+0000 7f7cb1e5f640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-11T07:59:37.849843+0000 front 2024-09-11T07:59:37.849707+0000 (oldest deadline 2024-09-11T08:00:02.549536+0000) 2024-09-11T08:00:21.749 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 08:00:21 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6[40580]: 2024-09-11T08:00:21.528+0000 7f7cb1e5f640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.110:6822 osd.2 since back 2024-09-11T07:59:47.051305+0000 front 2024-09-11T07:59:47.051207+0000 (oldest deadline 2024-09-11T08:00:11.151093+0000) 2024-09-11T08:00:21.749 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 08:00:21 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6[40580]: 2024-09-11T08:00:21.528+0000 7f7cb1e5f640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.110:6830 osd.3 since back 2024-09-11T07:59:55.251914+0000 front 2024-09-11T07:59:55.251881+0000 (oldest deadline 2024-09-11T08:00:21.151843+0000) 2024-09-11T08:00:21.749 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:21 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:21.510+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:29.150096+0000 front 2024-09-11T07:59:29.149948+0000 (oldest deadline 2024-09-11T07:59:55.049814+0000) 2024-09-11T08:00:21.749 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:21 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:21.510+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-11T07:59:37.251440+0000 front 2024-09-11T07:59:37.251347+0000 (oldest deadline 2024-09-11T08:00:03.151225+0000) 2024-09-11T08:00:21.749 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:21 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:21.510+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6822 osd.2 since back 2024-09-11T07:59:46.052137+0000 front 2024-09-11T07:59:46.052023+0000 (oldest deadline 2024-09-11T08:00:10.751990+0000) 2024-09-11T08:00:21.749 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:21 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:21.510+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6830 osd.3 since back 2024-09-11T07:59:55.453022+0000 front 2024-09-11T07:59:55.453001+0000 (oldest deadline 2024-09-11T08:00:20.752947+0000) 2024-09-11T08:00:22.727 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:22 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:22.461+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:29.150096+0000 front 2024-09-11T07:59:29.149948+0000 (oldest deadline 2024-09-11T07:59:55.049814+0000) 2024-09-11T08:00:22.727 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:22 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:22.461+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-11T07:59:37.251440+0000 front 2024-09-11T07:59:37.251347+0000 (oldest deadline 2024-09-11T08:00:03.151225+0000) 2024-09-11T08:00:22.727 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:22 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:22.461+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6822 osd.2 since back 2024-09-11T07:59:46.052137+0000 front 2024-09-11T07:59:46.052023+0000 (oldest deadline 2024-09-11T08:00:10.751990+0000) 2024-09-11T08:00:22.727 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:22 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:22.461+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6830 osd.3 since back 2024-09-11T07:59:55.453022+0000 front 2024-09-11T07:59:55.453001+0000 (oldest deadline 2024-09-11T08:00:20.752947+0000) 2024-09-11T08:00:22.998 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 08:00:22 smithi139 podman[56140]: 2024-09-11 08:00:22.814305706 +0000 UTC m=+5.525552501 container died 568d33525e1c577377084f003ae626bcaca5dd68dee3c8d97fc7f34738be3042 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-11T08:00:23.498 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 08:00:23 smithi139 podman[56140]: 2024-09-11 08:00:23.178573441 +0000 UTC m=+5.889820235 container cleanup 568d33525e1c577377084f003ae626bcaca5dd68dee3c8d97fc7f34738be3042 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-11T08:00:23.499 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 08:00:23 smithi139 bash[56140]: ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6 2024-09-11T08:00:23.941 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 08:00:23 smithi139 podman[56778]: 2024-09-11 08:00:23.756254859 +0000 UTC m=+1.023694856 container remove 568d33525e1c577377084f003ae626bcaca5dd68dee3c8d97fc7f34738be3042 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True) 2024-09-11T08:00:23.942 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:23 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:23.505+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:29.150096+0000 front 2024-09-11T07:59:29.149948+0000 (oldest deadline 2024-09-11T07:59:55.049814+0000) 2024-09-11T08:00:23.942 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:23 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:23.506+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-11T07:59:37.251440+0000 front 2024-09-11T07:59:37.251347+0000 (oldest deadline 2024-09-11T08:00:03.151225+0000) 2024-09-11T08:00:23.942 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:23 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:23.506+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6822 osd.2 since back 2024-09-11T07:59:46.052137+0000 front 2024-09-11T07:59:46.052023+0000 (oldest deadline 2024-09-11T08:00:10.751990+0000) 2024-09-11T08:00:23.942 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:23 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:23.506+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6830 osd.3 since back 2024-09-11T07:59:55.453022+0000 front 2024-09-11T07:59:55.453001+0000 (oldest deadline 2024-09-11T08:00:20.752947+0000) 2024-09-11T08:00:24.848 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:24 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:24.531+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:29.150096+0000 front 2024-09-11T07:59:29.149948+0000 (oldest deadline 2024-09-11T07:59:55.049814+0000) 2024-09-11T08:00:24.848 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:24 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:24.531+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-11T07:59:37.251440+0000 front 2024-09-11T07:59:37.251347+0000 (oldest deadline 2024-09-11T08:00:03.151225+0000) 2024-09-11T08:00:24.848 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:24 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:24.531+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6822 osd.2 since back 2024-09-11T07:59:46.052137+0000 front 2024-09-11T07:59:46.052023+0000 (oldest deadline 2024-09-11T08:00:10.751990+0000) 2024-09-11T08:00:24.848 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:24 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:24.531+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6830 osd.3 since back 2024-09-11T07:59:55.453022+0000 front 2024-09-11T07:59:55.453001+0000 (oldest deadline 2024-09-11T08:00:20.752947+0000) 2024-09-11T08:00:25.249 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 08:00:24 smithi139 podman[57020]: 2024-09-11 08:00:24.751610413 +0000 UTC m=+0.099202337 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T08:00:25.249 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 08:00:24 smithi139 podman[57020]: 2024-09-11 08:00:24.878263925 +0000 UTC m=+0.225855840 container create fe16ef91bcac09ac524533146bef7c4d4114103106be0618aa37e7e55bef4099 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-11T08:00:25.690 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 08:00:25 smithi139 podman[57020]: 2024-09-11 08:00:25.457323852 +0000 UTC m=+0.804915763 container init fe16ef91bcac09ac524533146bef7c4d4114103106be0618aa37e7e55bef4099 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=) 2024-09-11T08:00:25.690 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 08:00:25 smithi139 podman[57020]: 2024-09-11 08:00:25.46068243 +0000 UTC m=+0.808274341 container start fe16ef91bcac09ac524533146bef7c4d4114103106be0618aa37e7e55bef4099 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T08:00:25.691 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:25.496+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:29.150096+0000 front 2024-09-11T07:59:29.149948+0000 (oldest deadline 2024-09-11T07:59:55.049814+0000) 2024-09-11T08:00:25.691 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:25.496+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-11T07:59:37.251440+0000 front 2024-09-11T07:59:37.251347+0000 (oldest deadline 2024-09-11T08:00:03.151225+0000) 2024-09-11T08:00:25.691 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:25.496+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6822 osd.2 since back 2024-09-11T07:59:46.052137+0000 front 2024-09-11T07:59:46.052023+0000 (oldest deadline 2024-09-11T08:00:10.751990+0000) 2024-09-11T08:00:25.691 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:25 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:25.496+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6830 osd.3 since back 2024-09-11T07:59:55.453022+0000 front 2024-09-11T07:59:55.453001+0000 (oldest deadline 2024-09-11T08:00:20.752947+0000) 2024-09-11T08:00:25.986 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 08:00:25 smithi139 podman[57020]: 2024-09-11 08:00:25.699097717 +0000 UTC m=+1.046689636 container attach fe16ef91bcac09ac524533146bef7c4d4114103106be0618aa37e7e55bef4099 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2) 2024-09-11T08:00:25.986 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 11 08:00:25 smithi139 podman[57020]: 2024-09-11 08:00:25.699384974 +0000 UTC m=+1.046976889 container died fe16ef91bcac09ac524533146bef7c4d4114103106be0618aa37e7e55bef4099 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-6-deactivate, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-11T08:00:26.383 DEBUG:teuthology.orchestra.run.smithi139:> sudo pkill -f 'journalctl -f -n 0 -u ceph-41bd1096-7012-11ef-bcea-c7b262605968@osd.6.service' 2024-09-11T08:00:26.426 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:00:26.426 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-11T08:00:26.426 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-11T08:00:26.427 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ceph-41bd1096-7012-11ef-bcea-c7b262605968@osd.7 2024-09-11T08:00:26.749 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:26 smithi139 systemd[1]: Stopping Ceph osd.7 for 41bd1096-7012-11ef-bcea-c7b262605968... 2024-09-11T08:00:26.749 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:26.545+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:29.150096+0000 front 2024-09-11T07:59:29.149948+0000 (oldest deadline 2024-09-11T07:59:55.049814+0000) 2024-09-11T08:00:26.749 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:26.545+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-11T07:59:37.251440+0000 front 2024-09-11T07:59:37.251347+0000 (oldest deadline 2024-09-11T08:00:03.151225+0000) 2024-09-11T08:00:26.749 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:26.545+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6822 osd.2 since back 2024-09-11T07:59:46.052137+0000 front 2024-09-11T07:59:46.052023+0000 (oldest deadline 2024-09-11T08:00:10.751990+0000) 2024-09-11T08:00:26.749 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:26.545+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6830 osd.3 since back 2024-09-11T07:59:55.453022+0000 front 2024-09-11T07:59:55.453001+0000 (oldest deadline 2024-09-11T08:00:20.752947+0000) 2024-09-11T08:00:27.070 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:26.757+0000 7fdbae9d7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:00:27.070 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:26.757+0000 7fdbae9d7640 -1 osd.7 71 *** Got signal Terminated *** 2024-09-11T08:00:27.070 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:26 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:26.757+0000 7fdbae9d7640 -1 osd.7 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T08:00:27.998 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:27 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:27.588+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:29.150096+0000 front 2024-09-11T07:59:29.149948+0000 (oldest deadline 2024-09-11T07:59:55.049814+0000) 2024-09-11T08:00:27.999 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:27 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:27.588+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-11T07:59:37.251440+0000 front 2024-09-11T07:59:37.251347+0000 (oldest deadline 2024-09-11T08:00:03.151225+0000) 2024-09-11T08:00:27.999 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:27 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:27.588+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6822 osd.2 since back 2024-09-11T07:59:46.052137+0000 front 2024-09-11T07:59:46.052023+0000 (oldest deadline 2024-09-11T08:00:10.751990+0000) 2024-09-11T08:00:27.999 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:27 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:27.588+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6830 osd.3 since back 2024-09-11T07:59:55.453022+0000 front 2024-09-11T07:59:55.453001+0000 (oldest deadline 2024-09-11T08:00:20.752947+0000) 2024-09-11T08:00:28.999 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:28 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:28.611+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:29.150096+0000 front 2024-09-11T07:59:29.149948+0000 (oldest deadline 2024-09-11T07:59:55.049814+0000) 2024-09-11T08:00:28.999 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:28 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:28.611+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-11T07:59:37.251440+0000 front 2024-09-11T07:59:37.251347+0000 (oldest deadline 2024-09-11T08:00:03.151225+0000) 2024-09-11T08:00:28.999 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:28 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:28.611+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6822 osd.2 since back 2024-09-11T07:59:46.052137+0000 front 2024-09-11T07:59:46.052023+0000 (oldest deadline 2024-09-11T08:00:10.751990+0000) 2024-09-11T08:00:29.000 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:28 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:28.611+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6830 osd.3 since back 2024-09-11T07:59:55.453022+0000 front 2024-09-11T07:59:55.453001+0000 (oldest deadline 2024-09-11T08:00:20.752947+0000) 2024-09-11T08:00:29.999 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:29 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:29.648+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:29.150096+0000 front 2024-09-11T07:59:29.149948+0000 (oldest deadline 2024-09-11T07:59:55.049814+0000) 2024-09-11T08:00:29.999 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:29 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:29.648+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-11T07:59:37.251440+0000 front 2024-09-11T07:59:37.251347+0000 (oldest deadline 2024-09-11T08:00:03.151225+0000) 2024-09-11T08:00:29.999 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:29 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:29.648+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6822 osd.2 since back 2024-09-11T07:59:46.052137+0000 front 2024-09-11T07:59:46.052023+0000 (oldest deadline 2024-09-11T08:00:10.751990+0000) 2024-09-11T08:00:29.999 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:29 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:29.648+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6830 osd.3 since back 2024-09-11T07:59:55.453022+0000 front 2024-09-11T07:59:55.453001+0000 (oldest deadline 2024-09-11T08:00:20.752947+0000) 2024-09-11T08:00:30.999 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:30 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:30.630+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:29.150096+0000 front 2024-09-11T07:59:29.149948+0000 (oldest deadline 2024-09-11T07:59:55.049814+0000) 2024-09-11T08:00:30.999 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:30 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:30.630+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-11T07:59:37.251440+0000 front 2024-09-11T07:59:37.251347+0000 (oldest deadline 2024-09-11T08:00:03.151225+0000) 2024-09-11T08:00:30.999 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:30 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:30.630+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6822 osd.2 since back 2024-09-11T07:59:46.052137+0000 front 2024-09-11T07:59:46.052023+0000 (oldest deadline 2024-09-11T08:00:10.751990+0000) 2024-09-11T08:00:30.999 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:30 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:30.630+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6830 osd.3 since back 2024-09-11T07:59:55.453022+0000 front 2024-09-11T07:59:55.453001+0000 (oldest deadline 2024-09-11T08:00:20.752947+0000) 2024-09-11T08:00:31.999 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:31 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:31.608+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-11T07:59:29.150096+0000 front 2024-09-11T07:59:29.149948+0000 (oldest deadline 2024-09-11T07:59:55.049814+0000) 2024-09-11T08:00:31.999 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:31 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:31.608+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-11T07:59:37.251440+0000 front 2024-09-11T07:59:37.251347+0000 (oldest deadline 2024-09-11T08:00:03.151225+0000) 2024-09-11T08:00:31.999 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:31 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:31.608+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6822 osd.2 since back 2024-09-11T07:59:46.052137+0000 front 2024-09-11T07:59:46.052023+0000 (oldest deadline 2024-09-11T08:00:10.751990+0000) 2024-09-11T08:00:31.999 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:31 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7[44353]: 2024-09-11T08:00:31.608+0000 7fdbabff1640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.110:6830 osd.3 since back 2024-09-11T07:59:55.453022+0000 front 2024-09-11T07:59:55.453001+0000 (oldest deadline 2024-09-11T08:00:20.752947+0000) 2024-09-11T08:00:32.483 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:32 smithi139 podman[57162]: 2024-09-11 08:00:32.10130688 +0000 UTC m=+5.457066660 container died a6a947f35e559abd50255c09ed514c12cae4041479f87ff6a3044b634ae6079c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-11T08:00:32.748 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:32 smithi139 podman[57162]: 2024-09-11 08:00:32.482143356 +0000 UTC m=+5.837903136 container cleanup a6a947f35e559abd50255c09ed514c12cae4041479f87ff6a3044b634ae6079c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T08:00:32.748 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:32 smithi139 bash[57162]: ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7 2024-09-11T08:00:32.749 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:32 smithi139 podman[57176]: 2024-09-11 08:00:32.725033405 +0000 UTC m=+0.616037205 container remove a6a947f35e559abd50255c09ed514c12cae4041479f87ff6a3044b634ae6079c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-11T08:00:33.354 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:33 smithi139 podman[57271]: 2024-09-11 08:00:33.042518089 +0000 UTC m=+0.107400878 container create 84d6f95dfea6163a06a6306405d1edced2a482f6d4d9a992c41316ddfc9e9ac4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T08:00:33.354 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:33 smithi139 podman[57271]: 2024-09-11 08:00:32.950704847 +0000 UTC m=+0.015587598 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T08:00:33.354 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:33 smithi139 podman[57271]: 2024-09-11 08:00:33.269939091 +0000 UTC m=+0.334821835 container init 84d6f95dfea6163a06a6306405d1edced2a482f6d4d9a992c41316ddfc9e9ac4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-11T08:00:33.354 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:33 smithi139 podman[57271]: 2024-09-11 08:00:33.273181487 +0000 UTC m=+0.338064232 container start 84d6f95dfea6163a06a6306405d1edced2a482f6d4d9a992c41316ddfc9e9ac4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0) 2024-09-11T08:00:33.354 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:33 smithi139 podman[57271]: 2024-09-11 08:00:33.353229703 +0000 UTC m=+0.418112448 container attach 84d6f95dfea6163a06a6306405d1edced2a482f6d4d9a992c41316ddfc9e9ac4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T08:00:33.748 INFO:journalctl@ceph.osd.7.smithi139.stdout:Sep 11 08:00:33 smithi139 podman[57271]: 2024-09-11 08:00:33.456178041 +0000 UTC m=+0.521060785 container died 84d6f95dfea6163a06a6306405d1edced2a482f6d4d9a992c41316ddfc9e9ac4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T08:00:34.111 DEBUG:teuthology.orchestra.run.smithi139:> sudo pkill -f 'journalctl -f -n 0 -u ceph-41bd1096-7012-11ef-bcea-c7b262605968@osd.7.service' 2024-09-11T08:00:34.151 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:00:34.152 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-11T08:00:34.152 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopping rgw.foo.a... 2024-09-11T08:00:34.152 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-41bd1096-7012-11ef-bcea-c7b262605968@rgw.foo.a 2024-09-11T08:00:34.468 INFO:journalctl@ceph.rgw.foo.a.smithi110.stdout:Sep 11 08:00:34 smithi110 systemd[1]: Stopping Ceph rgw.foo.a for 41bd1096-7012-11ef-bcea-c7b262605968... 2024-09-11T08:00:34.718 INFO:journalctl@ceph.rgw.foo.a.smithi110.stdout:Sep 11 08:00:34 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-rgw-foo-a[50346]: 2024-09-11T08:00:34.464+0000 7f4aba58c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/radosgw -n client.rgw.foo.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:00:34.718 INFO:journalctl@ceph.rgw.foo.a.smithi110.stdout:Sep 11 08:00:34 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-rgw-foo-a[50346]: 2024-09-11T08:00:34.464+0000 7f4abe2937c0 -1 shutting down 2024-09-11T08:00:44.718 INFO:journalctl@ceph.rgw.foo.a.smithi110.stdout:Sep 11 08:00:44 smithi110 bash[64901]: time="2024-09-11T08:00:44Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-41bd1096-7012-11ef-bcea-c7b262605968-rgw-foo-a in 10 seconds, resorting to SIGKILL" 2024-09-11T08:00:44.718 INFO:journalctl@ceph.rgw.foo.a.smithi110.stdout:Sep 11 08:00:44 smithi110 podman[64901]: 2024-09-11 08:00:44.477839146 +0000 UTC m=+10.121623924 container died 3cb6994aa1dd473e78c075de2f846aa74f226a7ae09293a701bff5edf00d3929 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-rgw-foo-a, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2) 2024-09-11T08:00:45.114 INFO:journalctl@ceph.rgw.foo.a.smithi110.stdout:Sep 11 08:00:44 smithi110 podman[64901]: 2024-09-11 08:00:44.837869346 +0000 UTC m=+10.481654115 container cleanup 3cb6994aa1dd473e78c075de2f846aa74f226a7ae09293a701bff5edf00d3929 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-rgw-foo-a, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-11T08:00:45.114 INFO:journalctl@ceph.rgw.foo.a.smithi110.stdout:Sep 11 08:00:44 smithi110 bash[64901]: ceph-41bd1096-7012-11ef-bcea-c7b262605968-rgw-foo-a 2024-09-11T08:00:45.290 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-41bd1096-7012-11ef-bcea-c7b262605968@rgw.foo.a.service' 2024-09-11T08:00:45.327 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:00:45.327 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopped rgw.foo.a 2024-09-11T08:00:45.327 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-09-11T08:00:45.327 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ceph-41bd1096-7012-11ef-bcea-c7b262605968@prometheus.a 2024-09-11T08:00:45.638 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 08:00:45 smithi139 systemd[1]: Stopping Ceph prometheus.a for 41bd1096-7012-11ef-bcea-c7b262605968... 2024-09-11T08:00:46.249 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 08:00:45 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a[53449]: ts=2024-09-11T08:00:45.983Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-11T08:00:46.249 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 08:00:45 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a[53449]: ts=2024-09-11T08:00:45.983Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-09-11T08:00:46.249 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 08:00:45 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a[53449]: ts=2024-09-11T08:00:45.983Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-09-11T08:00:46.249 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 08:00:45 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a[53449]: ts=2024-09-11T08:00:45.983Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-11T08:00:46.249 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 08:00:45 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a[53449]: ts=2024-09-11T08:00:45.983Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-09-11T08:00:46.249 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 08:00:45 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a[53449]: ts=2024-09-11T08:00:45.983Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-09-11T08:00:46.249 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 08:00:45 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a[53449]: ts=2024-09-11T08:00:45.983Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-09-11T08:00:46.249 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 08:00:45 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a[53449]: ts=2024-09-11T08:00:45.983Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-09-11T08:00:46.249 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 08:00:45 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a[53449]: ts=2024-09-11T08:00:45.983Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-09-11T08:00:46.250 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 08:00:45 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a[53449]: ts=2024-09-11T08:00:45.984Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-09-11T08:00:46.250 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 08:00:45 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a[53449]: ts=2024-09-11T08:00:45.984Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-09-11T08:00:46.250 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 08:00:45 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a[53449]: ts=2024-09-11T08:00:45.984Z caller=main.go:1147 level=info msg="See you next time!" 2024-09-11T08:00:46.250 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 08:00:45 smithi139 podman[58162]: 2024-09-11 08:00:45.993809546 +0000 UTC m=+0.110735106 container died e9f19b573e38df547ffee6d8a3ef058c946c24e9ccdf3a9a3509897545bad3cf (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T08:00:46.618 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 08:00:46 smithi139 podman[58162]: 2024-09-11 08:00:46.432968894 +0000 UTC m=+0.549894459 container cleanup e9f19b573e38df547ffee6d8a3ef058c946c24e9ccdf3a9a3509897545bad3cf (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T08:00:46.618 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 08:00:46 smithi139 bash[58162]: ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a 2024-09-11T08:00:46.848 DEBUG:teuthology.orchestra.run.smithi139:> sudo pkill -f 'journalctl -f -n 0 -u ceph-41bd1096-7012-11ef-bcea-c7b262605968@prometheus.a.service' 2024-09-11T08:00:46.869 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 08:00:46 smithi139 podman[58190]: 2024-09-11 08:00:46.617709289 +0000 UTC m=+0.625924906 container remove e9f19b573e38df547ffee6d8a3ef058c946c24e9ccdf3a9a3509897545bad3cf (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T08:00:46.869 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 08:00:46 smithi139 systemd[1]: ceph-41bd1096-7012-11ef-bcea-c7b262605968@prometheus.a.service: Deactivated successfully. 2024-09-11T08:00:46.869 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 08:00:46 smithi139 systemd[1]: Stopped Ceph prometheus.a for 41bd1096-7012-11ef-bcea-c7b262605968. 2024-09-11T08:00:46.869 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Sep 11 08:00:46 smithi139 systemd[1]: ceph-41bd1096-7012-11ef-bcea-c7b262605968@prometheus.a.service: Consumed 1.279s CPU time. 2024-09-11T08:00:46.884 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:00:46.884 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-09-11T08:00:46.884 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 41bd1096-7012-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-11T08:00:49.153 INFO:journalctl@ceph.alertmanager.a.smithi110.stdout:Sep 11 08:00:48 smithi110 systemd[1]: Stopping Ceph alertmanager.a for 41bd1096-7012-11ef-bcea-c7b262605968... 2024-09-11T08:00:49.153 INFO:journalctl@ceph.alertmanager.a.smithi110.stdout:Sep 11 08:00:49 smithi110 ceph-41bd1096-7012-11ef-bcea-c7b262605968-alertmanager-a[59229]: ts=2024-09-11T08:00:49.132Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-11T08:00:49.457 INFO:journalctl@ceph.alertmanager.a.smithi110.stdout:Sep 11 08:00:49 smithi110 podman[65927]: 2024-09-11 08:00:49.152628293 +0000 UTC m=+0.125208789 container died 304e26c9eb31f6ef695600568713c624c5a05392b2f3cbbbb8426ae43318cfce (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T08:00:49.718 INFO:journalctl@ceph.alertmanager.a.smithi110.stdout:Sep 11 08:00:49 smithi110 podman[65927]: 2024-09-11 08:00:49.665871689 +0000 UTC m=+0.638452217 container cleanup 304e26c9eb31f6ef695600568713c624c5a05392b2f3cbbbb8426ae43318cfce (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T08:00:49.718 INFO:journalctl@ceph.alertmanager.a.smithi110.stdout:Sep 11 08:00:49 smithi110 bash[65927]: ceph-41bd1096-7012-11ef-bcea-c7b262605968-alertmanager-a 2024-09-11T08:00:50.121 INFO:journalctl@ceph.alertmanager.a.smithi110.stdout:Sep 11 08:00:49 smithi110 podman[65941]: 2024-09-11 08:00:49.833670927 +0000 UTC m=+0.677239951 container remove 304e26c9eb31f6ef695600568713c624c5a05392b2f3cbbbb8426ae43318cfce (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T08:00:50.121 INFO:journalctl@ceph.alertmanager.a.smithi110.stdout:Sep 11 08:00:49 smithi110 podman[65941]: 2024-09-11 08:00:49.892474343 +0000 UTC m=+0.736043367 volume remove 1a5d84506006c460716148e8a9c36d7a3b80e5c0ab0cb7c759e7196db920c8bc 2024-09-11T08:00:50.468 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 08:00:50 smithi110 systemd[1]: Stopping Ceph node-exporter.a for 41bd1096-7012-11ef-bcea-c7b262605968... 2024-09-11T08:00:50.468 INFO:journalctl@ceph.alertmanager.a.smithi110.stdout:Sep 11 08:00:50 smithi110 systemd[1]: ceph-41bd1096-7012-11ef-bcea-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-09-11T08:00:50.468 INFO:journalctl@ceph.alertmanager.a.smithi110.stdout:Sep 11 08:00:50 smithi110 systemd[1]: Stopped Ceph alertmanager.a for 41bd1096-7012-11ef-bcea-c7b262605968. 2024-09-11T08:00:50.837 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 08:00:50 smithi110 podman[66046]: 2024-09-11 08:00:50.572913866 +0000 UTC m=+0.108255730 container died 8c994666edb7f8440c3afff4fba58d55c427e31c5396dfe8d79f21cdf2966119 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-11T08:00:51.218 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 08:00:50 smithi110 podman[66046]: 2024-09-11 08:00:50.936800542 +0000 UTC m=+0.472142411 container cleanup 8c994666edb7f8440c3afff4fba58d55c427e31c5396dfe8d79f21cdf2966119 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-11T08:00:51.218 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 08:00:50 smithi110 bash[66046]: ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-a 2024-09-11T08:00:51.507 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 08:00:51 smithi110 podman[66059]: 2024-09-11 08:00:51.221485575 +0000 UTC m=+0.652588153 container remove 8c994666edb7f8440c3afff4fba58d55c427e31c5396dfe8d79f21cdf2966119 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-11T08:00:51.508 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 08:00:51 smithi110 systemd[1]: ceph-41bd1096-7012-11ef-bcea-c7b262605968@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-09-11T08:00:51.508 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 08:00:51 smithi110 systemd[1]: ceph-41bd1096-7012-11ef-bcea-c7b262605968@node-exporter.a.service: Failed with result 'exit-code'. 2024-09-11T08:00:51.508 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 08:00:51 smithi110 systemd[1]: Stopped Ceph node-exporter.a for 41bd1096-7012-11ef-bcea-c7b262605968. 2024-09-11T08:00:51.508 INFO:journalctl@ceph.node-exporter.a.smithi110.stdout:Sep 11 08:00:51 smithi110 systemd[1]: ceph-41bd1096-7012-11ef-bcea-c7b262605968@node-exporter.a.service: Consumed 1.892s CPU time. 2024-09-11T08:00:52.029 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 41bd1096-7012-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-11T08:00:53.998 INFO:journalctl@ceph.iscsi.iscsi.a.smithi139.stdout:Sep 11 08:00:53 smithi139 systemd[1]: Stopping Ceph iscsi.iscsi.a for 41bd1096-7012-11ef-bcea-c7b262605968... 2024-09-11T08:00:53.999 INFO:journalctl@ceph.iscsi.iscsi.a.smithi139.stdout:Sep 11 08:00:53 smithi139 ceph-41bd1096-7012-11ef-bcea-c7b262605968-iscsi-iscsi-a[46755]: debug Shutdown received 2024-09-11T08:01:04.163 INFO:journalctl@ceph.iscsi.iscsi.a.smithi139.stdout:Sep 11 08:01:03 smithi139 bash[58587]: time="2024-09-11T08:01:03Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-41bd1096-7012-11ef-bcea-c7b262605968-iscsi-iscsi-a in 10 seconds, resorting to SIGKILL" 2024-09-11T08:01:04.163 INFO:journalctl@ceph.iscsi.iscsi.a.smithi139.stdout:Sep 11 08:01:03 smithi139 podman[58587]: 2024-09-11 08:01:03.897340597 +0000 UTC m=+10.109260386 container died 0a249abecce95c0131e818145411b791f944a35874d09c69a76ea8186032f313 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-iscsi-iscsi-a, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T08:01:04.498 INFO:journalctl@ceph.iscsi.iscsi.a.smithi139.stdout:Sep 11 08:01:04 smithi139 podman[58587]: 2024-09-11 08:01:04.344482342 +0000 UTC m=+10.556402140 container cleanup 0a249abecce95c0131e818145411b791f944a35874d09c69a76ea8186032f313 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-iscsi-iscsi-a, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T08:01:04.499 INFO:journalctl@ceph.iscsi.iscsi.a.smithi139.stdout:Sep 11 08:01:04 smithi139 bash[58587]: ceph-41bd1096-7012-11ef-bcea-c7b262605968-iscsi-iscsi-a 2024-09-11T08:01:04.998 INFO:journalctl@ceph.iscsi.iscsi.a.smithi139.stdout:Sep 11 08:01:04 smithi139 podman[58615]: 2024-09-11 08:01:04.687335313 +0000 UTC m=+0.789483195 container remove 0a249abecce95c0131e818145411b791f944a35874d09c69a76ea8186032f313 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-iscsi-iscsi-a, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T08:01:04.999 INFO:journalctl@ceph.iscsi.iscsi.a.smithi139.stdout:Sep 11 08:01:04 smithi139 systemd[1]: ceph-41bd1096-7012-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Main process exited, code=exited, status=137/n/a 2024-09-11T08:01:04.999 INFO:journalctl@ceph.iscsi.iscsi.a.smithi139.stdout:Sep 11 08:01:04 smithi139 podman[58660]: 2024-09-11 08:01:04.917492042 +0000 UTC m=+0.101321316 container died 0b960d67abc5b00df15d420264e6561b90867ed112c825ac471136bdc7e06fbf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-11T08:01:05.499 INFO:journalctl@ceph.iscsi.iscsi.a.smithi139.stdout:Sep 11 08:01:05 smithi139 podman[58660]: 2024-09-11 08:01:05.267164411 +0000 UTC m=+0.450993683 container cleanup 0b960d67abc5b00df15d420264e6561b90867ed112c825ac471136bdc7e06fbf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-11T08:01:05.499 INFO:journalctl@ceph.iscsi.iscsi.a.smithi139.stdout:Sep 11 08:01:05 smithi139 bash[58660]: ceph-41bd1096-7012-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu 2024-09-11T08:01:05.499 INFO:journalctl@ceph.iscsi.iscsi.a.smithi139.stdout:Sep 11 08:01:05 smithi139 systemd[1]: ceph-41bd1096-7012-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Failed with result 'exit-code'. 2024-09-11T08:01:05.499 INFO:journalctl@ceph.iscsi.iscsi.a.smithi139.stdout:Sep 11 08:01:05 smithi139 systemd[1]: ceph-41bd1096-7012-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Unit process 46698 (conmon) remains running after unit stopped. 2024-09-11T08:01:05.499 INFO:journalctl@ceph.iscsi.iscsi.a.smithi139.stdout:Sep 11 08:01:05 smithi139 systemd[1]: ceph-41bd1096-7012-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Unit process 58671 (podman) remains running after unit stopped. 2024-09-11T08:01:05.499 INFO:journalctl@ceph.iscsi.iscsi.a.smithi139.stdout:Sep 11 08:01:05 smithi139 systemd[1]: Stopped Ceph iscsi.iscsi.a for 41bd1096-7012-11ef-bcea-c7b262605968. 2024-09-11T08:01:05.499 INFO:journalctl@ceph.iscsi.iscsi.a.smithi139.stdout:Sep 11 08:01:05 smithi139 systemd[1]: ceph-41bd1096-7012-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Consumed 2.360s CPU time. 2024-09-11T08:01:05.953 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 08:01:05 smithi139 systemd[1]: Stopping Ceph grafana.a for 41bd1096-7012-11ef-bcea-c7b262605968... 2024-09-11T08:01:05.954 INFO:journalctl@ceph.iscsi.iscsi.a.smithi139.stdout:Sep 11 08:01:05 smithi139 podman[58671]: 2024-09-11 08:01:05.518741152 +0000 UTC m=+0.605143464 container remove 0b960d67abc5b00df15d420264e6561b90867ed112c825ac471136bdc7e06fbf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True) 2024-09-11T08:01:06.231 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 08:01:05 smithi139 podman[58793]: 2024-09-11 08:01:05.952331167 +0000 UTC m=+0.139266547 container died 551749459dfbadccadfbb90889faa9cfb1667d54f8f346406d3852108b42971a (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, version=8.5, io.k8s.display-name=Red Hat Universal Base Image 8, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.openshift.tags=base rhel8, summary=Grafana Container configured for Ceph mgr/dashboard integration, maintainer=Paul Cuzner , name=ubi8, io.openshift.expose-services=, vcs-type=git, description=Ceph Grafana Container, release=239.1651231664, vendor=Red Hat, Inc., io.buildah.version=1.32.0, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, distribution-scope=public, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., architecture=x86_64, build-date=2022-04-29T11:48:11.055897, com.redhat.component=ubi8-container, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56) 2024-09-11T08:01:06.498 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 08:01:06 smithi139 podman[58793]: 2024-09-11 08:01:06.396641194 +0000 UTC m=+0.583576612 container cleanup 551749459dfbadccadfbb90889faa9cfb1667d54f8f346406d3852108b42971a (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., release=239.1651231664, io.openshift.expose-services=, maintainer=Paul Cuzner , url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, description=Ceph Grafana Container, io.buildah.version=1.32.0, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, name=ubi8, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, architecture=x86_64, summary=Grafana Container configured for Ceph mgr/dashboard integration, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, distribution-scope=public, io.openshift.tags=base rhel8, io.k8s.display-name=Red Hat Universal Base Image 8, version=8.5, vcs-type=git, com.redhat.component=ubi8-container, build-date=2022-04-29T11:48:11.055897, vendor=Red Hat, Inc.) 2024-09-11T08:01:06.499 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 08:01:06 smithi139 bash[58793]: ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a 2024-09-11T08:01:06.999 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 08:01:06 smithi139 podman[58805]: 2024-09-11 08:01:06.681423836 +0000 UTC m=+0.726195194 container remove 551749459dfbadccadfbb90889faa9cfb1667d54f8f346406d3852108b42971a (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-grafana-a, io.buildah.version=1.32.0, build-date=2022-04-29T11:48:11.055897, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., release=239.1651231664, distribution-scope=public, io.k8s.display-name=Red Hat Universal Base Image 8, io.openshift.tags=base rhel8, maintainer=Paul Cuzner , vcs-type=git, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, version=8.5, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, vendor=Red Hat, Inc., architecture=x86_64, com.redhat.component=ubi8-container, summary=Grafana Container configured for Ceph mgr/dashboard integration, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, description=Ceph Grafana Container, io.openshift.expose-services=, name=ubi8, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com) 2024-09-11T08:01:06.999 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 08:01:06 smithi139 systemd[1]: ceph-41bd1096-7012-11ef-bcea-c7b262605968@grafana.a.service: Main process exited, code=exited, status=143/n/a 2024-09-11T08:01:06.999 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 08:01:06 smithi139 systemd[1]: ceph-41bd1096-7012-11ef-bcea-c7b262605968@grafana.a.service: Failed with result 'exit-code'. 2024-09-11T08:01:06.999 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 08:01:06 smithi139 systemd[1]: Stopped Ceph grafana.a for 41bd1096-7012-11ef-bcea-c7b262605968. 2024-09-11T08:01:06.999 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Sep 11 08:01:06 smithi139 systemd[1]: ceph-41bd1096-7012-11ef-bcea-c7b262605968@grafana.a.service: Consumed 4.654s CPU time. 2024-09-11T08:01:07.271 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 08:01:07 smithi139 systemd[1]: Stopping Ceph node-exporter.b for 41bd1096-7012-11ef-bcea-c7b262605968... 2024-09-11T08:01:07.597 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 08:01:07 smithi139 podman[58915]: 2024-09-11 08:01:07.270338863 +0000 UTC m=+0.112373972 container died 82df887f4df0ddc1d40364fc6b65894d6136d08040e45743647a366736d3e37b (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-11T08:01:07.597 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 08:01:07 smithi139 podman[58915]: 2024-09-11 08:01:07.597034928 +0000 UTC m=+0.439070027 container cleanup 82df887f4df0ddc1d40364fc6b65894d6136d08040e45743647a366736d3e37b (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-11T08:01:07.994 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 08:01:07 smithi139 bash[58915]: ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-b 2024-09-11T08:01:07.994 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 08:01:07 smithi139 podman[58928]: 2024-09-11 08:01:07.798403459 +0000 UTC m=+0.532073081 container remove 82df887f4df0ddc1d40364fc6b65894d6136d08040e45743647a366736d3e37b (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-41bd1096-7012-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-11T08:01:07.994 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 08:01:07 smithi139 systemd[1]: ceph-41bd1096-7012-11ef-bcea-c7b262605968@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-09-11T08:01:08.347 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 08:01:08 smithi139 systemd[1]: ceph-41bd1096-7012-11ef-bcea-c7b262605968@node-exporter.b.service: Failed with result 'exit-code'. 2024-09-11T08:01:08.347 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 08:01:08 smithi139 systemd[1]: Stopped Ceph node-exporter.b for 41bd1096-7012-11ef-bcea-c7b262605968. 2024-09-11T08:01:08.347 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Sep 11 08:01:08 smithi139 systemd[1]: ceph-41bd1096-7012-11ef-bcea-c7b262605968@node-exporter.b.service: Consumed 1.850s CPU time. 2024-09-11T08:01:08.677 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T08:01:08.712 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T08:01:08.739 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-11T08:01:08.741 DEBUG:teuthology.misc:Transferring archived files from smithi110:/var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901238/remote/smithi110/crash 2024-09-11T08:01:08.742 DEBUG:teuthology.orchestra.run.smithi110:> sudo tar c -f - -C /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/crash -- . 2024-09-11T08:01:08.798 INFO:teuthology.orchestra.run.smithi110.stderr:tar: /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-11T08:01:08.799 INFO:teuthology.orchestra.run.smithi110.stderr:tar: Error is not recoverable: exiting now 2024-09-11T08:01:08.801 DEBUG:teuthology.misc:Transferring archived files from smithi139:/var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901238/remote/smithi139/crash 2024-09-11T08:01:08.802 DEBUG:teuthology.orchestra.run.smithi139:> sudo tar c -f - -C /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/crash -- . 2024-09-11T08:01:08.845 INFO:teuthology.orchestra.run.smithi139.stderr:tar: /var/lib/ceph/41bd1096-7012-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-11T08:01:08.845 INFO:teuthology.orchestra.run.smithi139.stderr:tar: Error is not recoverable: exiting now 2024-09-11T08:01:08.846 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-11T08:01:08.847 DEBUG:teuthology.orchestra.run.smithi110:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/41bd1096-7012-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v MON_DOWN | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | head -n 1 2024-09-11T08:01:08.875 INFO:teuthology.orchestra.run.smithi110.stderr:grep: /var/log/ceph/41bd1096-7012-11ef-bcea-c7b262605968/ceph.log: No such file or directory 2024-09-11T08:01:08.877 INFO:tasks.cephadm:Compressing logs... 2024-09-11T08:01:08.877 DEBUG:teuthology.orchestra.run.smithi110:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-11T08:01:08.919 DEBUG:teuthology.orchestra.run.smithi139:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-11T08:01:08.944 INFO:teuthology.orchestra.run.smithi139.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/41bd1096-7012-11ef-bcea-c7b262605968/tcmu-runner.log 2024-09-11T08:01:08.944 INFO:teuthology.orchestra.run.smithi139.stderr:: No such file or directory 2024-09-11T08:01:08.944 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/41bd1096-7012-11ef-bcea-c7b262605968/ceph-osd.7.log 2024-09-11T08:01:08.944 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/41bd1096-7012-11ef-bcea-c7b262605968/tcmu-runner.log: gzip -5 --verbose -- /var/log/ceph/41bd1096-7012-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-11T08:01:08.945 INFO:teuthology.orchestra.run.smithi139.stderr: 63.5% -- replaced with /var/log/ceph/41bd1096-7012-11ef-bcea-c7b262605968/tcmu-runner.log.gz 2024-09-11T08:01:08.950 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/41bd1096-7012-11ef-bcea-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/41bd1096-7012-11ef-bcea-c7b262605968/ceph-osd.4.log 2024-09-11T08:01:08.950 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/41bd1096-7012-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/41bd1096-7012-11ef-bcea-c7b262605968/ceph-osd.6.log 2024-09-11T08:01:08.950 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/41bd1096-7012-11ef-bcea-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/41bd1096-7012-11ef-bcea-c7b262605968/ceph-osd.5.log 2024-09-11T08:01:08.950 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/41bd1096-7012-11ef-bcea-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/41bd1096-7012-11ef-bcea-c7b262605968/ceph-mgr.x.log 2024-09-11T08:01:08.950 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/41bd1096-7012-11ef-bcea-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/41bd1096-7012-11ef-bcea-c7b262605968/ceph-mon.b.log 2024-09-11T08:01:08.950 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/41bd1096-7012-11ef-bcea-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-11T08:01:08.950 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/41bd1096-7012-11ef-bcea-c7b262605968/ceph-mon.b.log: /var/log/ceph/cephadm.log: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-11T08:01:08.951 INFO:teuthology.orchestra.run.smithi110.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/41bd1096-7012-11ef-bcea-c7b262605968/ceph-osd.0.log 2024-09-11T08:01:08.951 INFO:teuthology.orchestra.run.smithi110.stderr:: No such file or directory 2024-09-11T08:01:08.951 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/41bd1096-7012-11ef-bcea-c7b262605968/ceph-osd.1.log 2024-09-11T08:01:08.951 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/41bd1096-7012-11ef-bcea-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/41bd1096-7012-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-11T08:01:08.951 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/41bd1096-7012-11ef-bcea-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/41bd1096-7012-11ef-bcea-c7b262605968/ceph-mon.c.log 2024-09-11T08:01:08.951 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/41bd1096-7012-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/41bd1096-7012-11ef-bcea-c7b262605968/ceph-mon.a.log 2024-09-11T08:01:08.951 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/41bd1096-7012-11ef-bcea-c7b262605968/ceph-osd.2.log 2024-09-11T08:01:08.952 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/41bd1096-7012-11ef-bcea-c7b262605968/ceph-mon.c.log: /var/log/ceph/41bd1096-7012-11ef-bcea-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/41bd1096-7012-11ef-bcea-c7b262605968/ceph-mgr.y.log 2024-09-11T08:01:08.952 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/41bd1096-7012-11ef-bcea-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/41bd1096-7012-11ef-bcea-c7b262605968/ceph-client.rgw.foo.a.log 2024-09-11T08:01:08.952 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/41bd1096-7012-11ef-bcea-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/41bd1096-7012-11ef-bcea-c7b262605968/ceph-osd.3.log 2024-09-11T08:01:08.952 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/41bd1096-7012-11ef-bcea-c7b262605968/ceph-client.rgw.foo.a.log: 58.8% -- replaced with /var/log/ceph/41bd1096-7012-11ef-bcea-c7b262605968/ceph-client.rgw.foo.a.log.gz 2024-09-11T08:01:08.952 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-11T08:01:08.953 INFO:teuthology.orchestra.run.smithi139.stderr: 90.8% -- replaced with /var/log/ceph/41bd1096-7012-11ef-bcea-c7b262605968/ceph-mgr.x.log.gz 2024-09-11T08:01:08.968 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/41bd1096-7012-11ef-bcea-c7b262605968/ceph-osd.3.log: /var/log/ceph/cephadm.log: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-11T08:01:08.995 INFO:teuthology.orchestra.run.smithi139.stderr: 90.3% -- replaced with /var/log/ceph/41bd1096-7012-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-11T08:01:09.001 INFO:teuthology.orchestra.run.smithi110.stderr: 90.2% -- replaced with /var/log/ceph/41bd1096-7012-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-11T08:01:09.062 INFO:teuthology.orchestra.run.smithi139.stderr: 92.3% -- replaced with /var/log/ceph/41bd1096-7012-11ef-bcea-c7b262605968/ceph-mon.b.log.gz 2024-09-11T08:01:09.087 INFO:teuthology.orchestra.run.smithi110.stderr: 92.2% -- replaced with /var/log/ceph/41bd1096-7012-11ef-bcea-c7b262605968/ceph-mon.c.log.gz 2024-09-11T08:01:09.097 INFO:teuthology.orchestra.run.smithi139.stderr: 93.1% -- replaced with /var/log/ceph/41bd1096-7012-11ef-bcea-c7b262605968/ceph-osd.6.log.gz 2024-09-11T08:01:09.110 INFO:teuthology.orchestra.run.smithi110.stderr: 93.2% -- replaced with /var/log/ceph/41bd1096-7012-11ef-bcea-c7b262605968/ceph-osd.2.log.gz 2024-09-11T08:01:09.116 INFO:teuthology.orchestra.run.smithi110.stderr: 87.5% -- replaced with /var/log/ceph/41bd1096-7012-11ef-bcea-c7b262605968/ceph-mgr.y.log.gz 2024-09-11T08:01:09.132 INFO:teuthology.orchestra.run.smithi139.stderr: 93.4% -- replaced with /var/log/ceph/41bd1096-7012-11ef-bcea-c7b262605968/ceph-osd.4.log.gz 2024-09-11T08:01:09.160 INFO:teuthology.orchestra.run.smithi139.stderr: 93.4% -- replaced with /var/log/ceph/41bd1096-7012-11ef-bcea-c7b262605968/ceph-osd.7.log.gz 2024-09-11T08:01:09.165 INFO:teuthology.orchestra.run.smithi139.stderr: 93.2% -- replaced with /var/log/ceph/41bd1096-7012-11ef-bcea-c7b262605968/ceph-osd.5.log.gz 2024-09-11T08:01:09.167 INFO:teuthology.orchestra.run.smithi139.stderr: 2024-09-11T08:01:09.167 INFO:teuthology.orchestra.run.smithi139.stderr:real 0m0.233s 2024-09-11T08:01:09.167 INFO:teuthology.orchestra.run.smithi139.stderr:user 0m0.925s 2024-09-11T08:01:09.167 INFO:teuthology.orchestra.run.smithi139.stderr:sys 0m0.050s 2024-09-11T08:01:09.197 INFO:teuthology.orchestra.run.smithi110.stderr: 93.3% -- replaced with /var/log/ceph/41bd1096-7012-11ef-bcea-c7b262605968/ceph-osd.0.log.gz 2024-09-11T08:01:09.210 INFO:teuthology.orchestra.run.smithi110.stderr: 93.5% -- replaced with /var/log/ceph/41bd1096-7012-11ef-bcea-c7b262605968/ceph-osd.3.log.gz 2024-09-11T08:01:09.225 INFO:teuthology.orchestra.run.smithi110.stderr: 93.5% -- replaced with /var/log/ceph/41bd1096-7012-11ef-bcea-c7b262605968/ceph-osd.1.log.gz 2024-09-11T08:01:09.303 INFO:teuthology.orchestra.run.smithi110.stderr: 91.3% -- replaced with /var/log/ceph/41bd1096-7012-11ef-bcea-c7b262605968/ceph-mon.a.log.gz 2024-09-11T08:01:09.305 INFO:teuthology.orchestra.run.smithi110.stderr: 2024-09-11T08:01:09.305 INFO:teuthology.orchestra.run.smithi110.stderr:real 0m0.373s 2024-09-11T08:01:09.305 INFO:teuthology.orchestra.run.smithi110.stderr:user 0m1.624s 2024-09-11T08:01:09.305 INFO:teuthology.orchestra.run.smithi110.stderr:sys 0m0.082s 2024-09-11T08:01:09.305 INFO:tasks.cephadm:Archiving logs... 2024-09-11T08:01:09.306 DEBUG:teuthology.misc:Transferring archived files from smithi110:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901238/remote/smithi110/log 2024-09-11T08:01:09.307 DEBUG:teuthology.orchestra.run.smithi110:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-11T08:01:09.590 DEBUG:teuthology.misc:Transferring archived files from smithi139:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901238/remote/smithi139/log 2024-09-11T08:01:09.591 DEBUG:teuthology.orchestra.run.smithi139:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-11T08:01:09.743 INFO:tasks.cephadm:Removing cluster... 2024-09-11T08:01:09.743 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 41bd1096-7012-11ef-bcea-c7b262605968 --force 2024-09-11T08:01:10.061 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 41bd1096-7012-11ef-bcea-c7b262605968 --force 2024-09-11T08:01:10.388 INFO:tasks.cephadm:Removing cephadm ... 2024-09-11T08:01:10.388 DEBUG:teuthology.orchestra.run.smithi110:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-11T08:01:10.405 DEBUG:teuthology.orchestra.run.smithi139:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-11T08:01:10.420 INFO:tasks.cephadm:Teardown complete 2024-09-11T08:01:10.421 DEBUG:teuthology.run_tasks:Unwinding manager nvme_loop 2024-09-11T08:01:10.435 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi110:/dev/vg_nvme/lv_1... 2024-09-11T08:01:10.435 DEBUG:teuthology.orchestra.run.smithi110:> sudo nvme disconnect -n lv_1 2024-09-11T08:01:10.610 INFO:teuthology.orchestra.run.smithi110.stdout:NQN:lv_1 disconnected 1 controller(s) 2024-09-11T08:01:10.613 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi110:/dev/vg_nvme/lv_2... 2024-09-11T08:01:10.613 DEBUG:teuthology.orchestra.run.smithi110:> sudo nvme disconnect -n lv_2 2024-09-11T08:01:10.767 INFO:teuthology.orchestra.run.smithi110.stdout:NQN:lv_2 disconnected 1 controller(s) 2024-09-11T08:01:10.768 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi110:/dev/vg_nvme/lv_3... 2024-09-11T08:01:10.768 DEBUG:teuthology.orchestra.run.smithi110:> sudo nvme disconnect -n lv_3 2024-09-11T08:01:10.904 INFO:teuthology.orchestra.run.smithi110.stdout:NQN:lv_3 disconnected 1 controller(s) 2024-09-11T08:01:10.905 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi110:/dev/vg_nvme/lv_4... 2024-09-11T08:01:10.905 DEBUG:teuthology.orchestra.run.smithi110:> sudo nvme disconnect -n lv_4 2024-09-11T08:01:11.054 INFO:teuthology.orchestra.run.smithi110.stdout:NQN:lv_4 disconnected 1 controller(s) 2024-09-11T08:01:11.056 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-11T08:01:11.056 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/scratch_devs 2024-09-11T08:01:11.080 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi139:/dev/vg_nvme/lv_1... 2024-09-11T08:01:11.081 DEBUG:teuthology.orchestra.run.smithi139:> sudo nvme disconnect -n lv_1 2024-09-11T08:01:11.240 INFO:teuthology.orchestra.run.smithi139.stdout:NQN:lv_1 disconnected 1 controller(s) 2024-09-11T08:01:11.242 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi139:/dev/vg_nvme/lv_2... 2024-09-11T08:01:11.242 DEBUG:teuthology.orchestra.run.smithi139:> sudo nvme disconnect -n lv_2 2024-09-11T08:01:11.393 INFO:teuthology.orchestra.run.smithi139.stdout:NQN:lv_2 disconnected 1 controller(s) 2024-09-11T08:01:11.395 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi139:/dev/vg_nvme/lv_3... 2024-09-11T08:01:11.395 DEBUG:teuthology.orchestra.run.smithi139:> sudo nvme disconnect -n lv_3 2024-09-11T08:01:11.578 INFO:teuthology.orchestra.run.smithi139.stdout:NQN:lv_3 disconnected 1 controller(s) 2024-09-11T08:01:11.580 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi139:/dev/vg_nvme/lv_4... 2024-09-11T08:01:11.580 DEBUG:teuthology.orchestra.run.smithi139:> sudo nvme disconnect -n lv_4 2024-09-11T08:01:11.724 INFO:teuthology.orchestra.run.smithi139.stdout:NQN:lv_4 disconnected 1 controller(s) 2024-09-11T08:01:11.726 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-11T08:01:11.726 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/scratch_devs 2024-09-11T08:01:11.750 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-11T08:01:11.759 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-11T08:01:11.760 DEBUG:teuthology.orchestra.run.smithi110:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T08:01:11.762 DEBUG:teuthology.orchestra.run.smithi139:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T08:01:11.777 INFO:teuthology.orchestra.run.smithi110.stderr:bash: line 1: ntpq: command not found 2024-09-11T08:01:11.782 INFO:teuthology.orchestra.run.smithi110.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T08:01:11.782 INFO:teuthology.orchestra.run.smithi110.stdout:=============================================================================== 2024-09-11T08:01:11.782 INFO:teuthology.orchestra.run.smithi110.stdout:^* hv01.front.sepia.ceph.com 3 6 377 45 -780us[ -779us] +/- 60ms 2024-09-11T08:01:11.782 INFO:teuthology.orchestra.run.smithi110.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 43 +592us[ +592us] +/- 53ms 2024-09-11T08:01:11.782 INFO:teuthology.orchestra.run.smithi110.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 46 +170us[ +170us] +/- 82ms 2024-09-11T08:01:11.782 INFO:teuthology.orchestra.run.smithi110.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T08:01:11.806 INFO:teuthology.orchestra.run.smithi139.stderr:bash: line 1: ntpq: command not found 2024-09-11T08:01:11.810 INFO:teuthology.orchestra.run.smithi139.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T08:01:11.810 INFO:teuthology.orchestra.run.smithi139.stdout:=============================================================================== 2024-09-11T08:01:11.810 INFO:teuthology.orchestra.run.smithi139.stdout:^* hv01.front.sepia.ceph.com 3 6 377 43 -793us[ -783us] +/- 61ms 2024-09-11T08:01:11.810 INFO:teuthology.orchestra.run.smithi139.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 46 +588us[ +598us] +/- 53ms 2024-09-11T08:01:11.810 INFO:teuthology.orchestra.run.smithi139.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 44 +173us[ +183us] +/- 82ms 2024-09-11T08:01:11.810 INFO:teuthology.orchestra.run.smithi139.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T08:01:11.811 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-11T08:01:11.819 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-11T08:01:11.820 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-11T08:01:11.879 DEBUG:teuthology.orchestra.run.smithi110:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-11T08:01:11.957 DEBUG:teuthology.orchestra.run.smithi139:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-11T08:01:12.024 DEBUG:teuthology.orchestra.run.smithi110:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-11T08:01:12.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:01:12.055 DEBUG:teuthology.orchestra.run.smithi139:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-11T08:01:12.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:01:12.083 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-11T08:01:12.092 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-11T08:01:12.116 INFO:teuthology.task.internal:Duration was 993.070734 seconds 2024-09-11T08:01:12.116 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-11T08:01:12.199 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-11T08:01:12.200 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-11T08:01:12.202 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-11T08:01:12.241 INFO:teuthology.orchestra.run.smithi139.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T08:01:12.241 INFO:teuthology.orchestra.run.smithi110.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T08:01:12.756 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-11T08:01:12.756 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi110.front.sepia.ceph.com 2024-09-11T08:01:12.757 DEBUG:teuthology.orchestra.run.smithi110:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-11T08:01:12.819 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi139.front.sepia.ceph.com 2024-09-11T08:01:12.819 DEBUG:teuthology.orchestra.run.smithi139:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-11T08:01:12.844 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-11T08:01:12.844 DEBUG:teuthology.orchestra.run.smithi110:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-11T08:01:12.861 DEBUG:teuthology.orchestra.run.smithi139:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-11T08:01:12.910 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-11T08:01:12.910 DEBUG:teuthology.orchestra.run.smithi110:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-11T08:01:12.928 DEBUG:teuthology.orchestra.run.smithi139:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-11T08:01:13.192 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-11T08:01:13.202 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-11T08:01:13.202 DEBUG:teuthology.orchestra.run.smithi110:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-11T08:01:13.226 DEBUG:teuthology.orchestra.run.smithi139:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-11T08:01:13.251 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-11T08:01:13.262 DEBUG:teuthology.orchestra.run.smithi110:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-11T08:01:13.269 DEBUG:teuthology.orchestra.run.smithi139:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-11T08:01:13.290 INFO:teuthology.orchestra.run.smithi110.stdout:kernel.core_pattern = core 2024-09-11T08:01:13.314 INFO:teuthology.orchestra.run.smithi139.stdout:kernel.core_pattern = core 2024-09-11T08:01:13.338 DEBUG:teuthology.orchestra.run.smithi110:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-11T08:01:13.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:01:13.369 DEBUG:teuthology.orchestra.run.smithi139:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-11T08:01:13.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:01:13.392 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-11T08:01:13.402 INFO:teuthology.task.internal:Transferring archived files... 2024-09-11T08:01:13.403 DEBUG:teuthology.misc:Transferring archived files from smithi110:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901238/remote/smithi110 2024-09-11T08:01:13.403 DEBUG:teuthology.orchestra.run.smithi110:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-11T08:01:13.449 DEBUG:teuthology.misc:Transferring archived files from smithi139:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901238/remote/smithi139 2024-09-11T08:01:13.450 DEBUG:teuthology.orchestra.run.smithi139:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-11T08:01:13.486 INFO:teuthology.task.internal:Removing archive directory... 2024-09-11T08:01:13.486 DEBUG:teuthology.orchestra.run.smithi110:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-11T08:01:13.489 DEBUG:teuthology.orchestra.run.smithi139:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-11T08:01:13.535 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-11T08:01:13.547 INFO:teuthology.task.internal:Not uploading archives. 2024-09-11T08:01:13.547 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-11T08:01:13.568 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-11T08:01:13.568 DEBUG:teuthology.orchestra.run.smithi110:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-11T08:01:13.571 DEBUG:teuthology.orchestra.run.smithi139:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-11T08:01:13.583 INFO:teuthology.orchestra.run.smithi110.stdout: 265336 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 11 08:01 /home/ubuntu/cephtest 2024-09-11T08:01:13.589 INFO:teuthology.orchestra.run.smithi139.stdout: 265846 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 11 08:01 /home/ubuntu/cephtest 2024-09-11T08:01:13.591 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-11T08:01:13.622 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-11T08:01:13.745 INFO:teuthology.run:Summary data: description: orch:cephadm/smoke/{0-distro/centos_9.stream 0-nvme-loop agent/on fixed-2 mon_election/classic start} duration: 993.0707340240479 owner: scheduled_gabrioux@teuthology success: true 2024-09-11T08:01:13.745 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-11T08:01:13.826 INFO:teuthology.run:pass